Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: let users pass sync/async functions for resolving option values for headers/params #2184

Merged
merged 11 commits into from
Dec 19, 2024
26 changes: 26 additions & 0 deletions .changeset/wicked-papayas-tease.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
"@electric-sql/client": patch
"@electric-sql/docs": patch
---

This PR adds support for function-based options in the TypeScript client's params and headers. Functions can be either synchronous or asynchronous and are resolved in parallel when needed.

```typescript
const stream = new ShapeStream({
url: 'http://localhost:3000/v1/shape',
params: {
table: 'items',
userId: () => getCurrentUserId(),
filter: async () => await getUserPreferences()
},
headers: {
'Authorization': async () => `Bearer ${await getAccessToken()}`
}
})
```

## Common Use Cases
- Authentication tokens that need to be refreshed
- User-specific parameters that may change
- Dynamic filtering based on current state
- Multi-tenant applications where context determines the request
2 changes: 1 addition & 1 deletion packages/typescript-client/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Real-time Postgres sync for modern apps.

Electric provides an [HTTP interface](https://electric-sql.com/docs/api/http) to Postgres to enable a massive number of clients to query and get real-time updates to subsets of the database, called [Shapes](https://electric-sql.com//docs/guides/shapes). In this way, Electric turns Postgres into a real-time database.

The TypeScript client helps ease reading Shapes from the HTTP API in the browser and other JavaScript environments, such as edge functions and server-side Node/Bun/Deno applications. It supports both fine-grained and coarse-grained reactivity patterns — you can subscribe to see every row that changes, or you can just subscribe to get the whole shape whenever it changes.
The TypeScript client helps ease reading Shapes from the HTTP API in the browser and other JavaScript environments, such as edge functions and server-side Node/Bun/Deno applications. It supports both fine-grained and coarse-grained reactivity patterns — you can subscribe to see every row that changes, or you can just subscribe to get the whole shape whenever it changes. The client also supports dynamic options through function-based params and headers, making it easy to handle auth tokens, user context, and other runtime values.

## Install

Expand Down
91 changes: 77 additions & 14 deletions packages/typescript-client/src/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,22 @@ type ReservedParamKeys =

/**
* External params type - what users provide.
* Includes documented PostgreSQL params and allows string or string[] values for any additional params.
* Includes documented PostgreSQL params and allows string, string[], or function values for any additional params.
*/
type ExternalParamsRecord = Partial<PostgresParams> & {
[K in string as K extends ReservedParamKeys ? never : K]: string | string[]
export type ExternalParamsRecord = PostgresParams & {
KyleAMathews marked this conversation as resolved.
Show resolved Hide resolved
[key: string]:
| string
| string[]
| (() => string | string[] | Promise<string | string[]>)
| undefined
}

/**
* External headers type - what users provide.
* Allows string or function values for any header.
*/
export type ExternalHeadersRecord = {
KyleAMathews marked this conversation as resolved.
Show resolved Hide resolved
[key: string]: string | (() => string | Promise<string>)
}

/**
Expand All @@ -103,19 +115,59 @@ type InternalParamsRecord = {
}

/**
* Helper function to convert external params to internal format
* Helper function to resolve a function or value to its final value
*/
function toInternalParams(params: ExternalParamsRecord): InternalParamsRecord {
const result: InternalParamsRecord = {}
for (const [key, value] of Object.entries(params)) {
result[key] = Array.isArray(value) ? value.join(`,`) : value
export async function resolveValue<T>(
KyleAMathews marked this conversation as resolved.
Show resolved Hide resolved
value: T | (() => T | Promise<T>)
): Promise<T> {
if (typeof value === `function`) {
return (value as () => T | Promise<T>)()
}
return result
return value
}

/**
* Helper function to convert external params to internal format
*/
async function toInternalParams(
params: ExternalParamsRecord
): Promise<InternalParamsRecord> {
const entries = Object.entries(params)
const resolvedEntries = await Promise.all(
entries.map(async ([key, value]) => {
if (value === undefined) return [key, undefined]
const resolvedValue = await resolveValue(value)
return [
key,
Array.isArray(resolvedValue) ? resolvedValue.join(`,`) : resolvedValue,
]
})
)

return Object.fromEntries(
resolvedEntries.filter(([_, value]) => value !== undefined)
)
}

/**
* Helper function to resolve headers
*/
async function resolveHeaders(
headers?: ExternalHeadersRecord
): Promise<Record<string, string>> {
if (!headers) return {}

const entries = Object.entries(headers)
const resolvedEntries = await Promise.all(
entries.map(async ([key, value]) => [key, await resolveValue(value)])
)

return Object.fromEntries(resolvedEntries)
}

type RetryOpts = {
params?: ExternalParamsRecord
headers?: Record<string, string>
headers?: ExternalHeadersRecord
}

type ShapeStreamErrorHandler = (
Expand Down Expand Up @@ -150,12 +202,18 @@ export interface ShapeStreamOptions<T = never> {

/**
* HTTP headers to attach to requests made by the client.
* Can be used for adding authentication headers.
* Values can be strings or functions (sync or async) that return strings.
* Function values are resolved in parallel when needed, making this useful
* for authentication tokens or other dynamic headers.
*/
headers?: Record<string, string>
headers?: ExternalHeadersRecord

/**
* Additional request parameters to attach to the URL.
* Values can be strings, string arrays, or functions (sync or async) that return these types.
* Function values are resolved in parallel when needed, making this useful
* for user-specific parameters or dynamic filters.
*
* These will be merged with Electric's standard parameters.
* Note: You cannot use Electric's reserved parameter names
* (offset, handle, live, cursor).
Expand Down Expand Up @@ -320,6 +378,9 @@ export class ShapeStream<T extends Row<unknown> = Row>

const fetchUrl = new URL(url)

// Resolve headers first
const requestHeaders = await resolveHeaders(this.options.headers)
KyleAMathews marked this conversation as resolved.
Show resolved Hide resolved

// Add any custom parameters first
if (this.options.params) {
// Check for reserved parameter names
Expand All @@ -332,8 +393,10 @@ export class ShapeStream<T extends Row<unknown> = Row>
)
}

// Resolve params
const params = await toInternalParams(this.options.params)
KyleAMathews marked this conversation as resolved.
Show resolved Hide resolved

// Add PostgreSQL-specific parameters from params
const params = toInternalParams(this.options.params)
if (params.table)
fetchUrl.searchParams.set(TABLE_QUERY_PARAM, params.table)
if (params.where)
Expand Down Expand Up @@ -381,7 +444,7 @@ export class ShapeStream<T extends Row<unknown> = Row>
try {
response = await this.#fetchClient(fetchUrl.toString(), {
signal,
headers: this.options.headers,
headers: requestHeaders,
})
this.#connected = true
} catch (e) {
Expand Down
49 changes: 48 additions & 1 deletion packages/typescript-client/test/client.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { testWithIssuesTable as it } from './support/test-context'
import { ShapeStream, Shape, FetchError } from '../src'
import { Message, Row, ChangeMessage } from '../src/types'
import { MissingHeadersError } from '../src/error'
import { resolveValue } from '../src'

const BASE_URL = inject(`baseUrl`)

Expand Down Expand Up @@ -188,7 +189,6 @@ describe(`Shape`, () => {
fetchClient: fetchWrapper,
})
const shape = new Shape(shapeStream)

let dataUpdateCount = 0
await new Promise<void>((resolve, reject) => {
setTimeout(() => reject(`Timed out waiting for data changes`), 1000)
Expand Down Expand Up @@ -664,4 +664,51 @@ describe(`Shape`, () => {
await clearIssuesShape(shapeStream.shapeHandle)
}
})

it(`should support function-based params and headers`, async ({
issuesTableUrl,
}) => {
const mockParamFn = vi.fn().mockReturnValue(`test-value`)
const mockAsyncParamFn = vi.fn().mockResolvedValue(`test-value`)
const mockHeaderFn = vi.fn().mockReturnValue(`test-value`)
const mockAsyncHeaderFn = vi.fn().mockResolvedValue(`test-value`)

// Test with synchronous functions
const shapeStream1 = new ShapeStream({
url: `${BASE_URL}/v1/shape`,
params: {
table: issuesTableUrl,
customParam: mockParamFn,
},
headers: {
'X-Custom-Header': mockHeaderFn,
},
})
const shape1 = new Shape(shapeStream1)
await shape1.value

expect(mockParamFn).toHaveBeenCalled()
expect(mockHeaderFn).toHaveBeenCalled()

// Test with async functions
const shapeStream2 = new ShapeStream({
url: `${BASE_URL}/v1/shape`,
params: {
table: issuesTableUrl,
customParam: mockAsyncParamFn,
},
headers: {
'X-Custom-Header': mockAsyncHeaderFn,
},
})
const shape2 = new Shape(shapeStream2)
await shape2.value

expect(mockAsyncParamFn).toHaveBeenCalled()
expect(mockAsyncHeaderFn).toHaveBeenCalled()

// Verify the resolved values
expect(await resolveValue(mockParamFn())).toBe(`test-value`)
expect(await resolveValue(mockAsyncParamFn())).toBe(`test-value`)
})
})
25 changes: 25 additions & 0 deletions website/docs/api/clients/typescript.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,31 @@ const stream = new ShapeStream({
})
```

#### Dynamic Options

Both `params` and `headers` support function options that are resolved when needed. These functions can be synchronous or asynchronous:

```typescript
const stream = new ShapeStream({
url: 'http://localhost:3000/v1/shape',
params: {
table: 'items',
userId: () => getCurrentUserId(),
filter: async () => await getUserPreferences()
},
headers: {
'Authorization': async () => `Bearer ${await getAccessToken()}`,
'X-Tenant-Id': () => getCurrentTenant()
}
})
```

Function options are resolved in parallel, making this pattern efficient for multiple async operations like fetching auth tokens and user context. Common use cases include:
- Authentication tokens that need to be refreshed
- User-specific parameters that may change
- Dynamic filtering based on current state
KyleAMathews marked this conversation as resolved.
Show resolved Hide resolved
- Multi-tenant applications where context determines the request

#### Messages

A `ShapeStream` consumes and emits a stream of messages. These messages can either be a `ChangeMessage` representing a change to the shape data:
Expand Down
22 changes: 22 additions & 0 deletions website/docs/guides/auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,28 @@ See the [./client](https://github.com/electric-sql/electric/tree/main/examples/g

<<< @../../examples/gatekeeper-auth/client/index.ts{typescript}

### Dynamic Auth Options

The TypeScript client supports function-based options for headers and params, making it easy to handle dynamic auth tokens:

```typescript
const stream = new ShapeStream({
url: 'http://localhost:3000/v1/shape',
headers: {
// Token will be refreshed on each request
'Authorization': async () => `Bearer ${await getAccessToken()}`
}
})
```

This pattern is particularly useful when:
- Your auth tokens need periodic refreshing
- You're using session-based authentication
- You need to fetch tokens from a secure storage
- You want to handle token rotation automatically

The function is called when needed and its value is resolved in parallel with other dynamic options, making it efficient for real-world auth scenarios.

## Notes

### External services
Expand Down
Loading