Skip to content

Commit

Permalink
Edits
Browse files Browse the repository at this point in the history
  • Loading branch information
samwillis committed Aug 6, 2024
1 parent 4b01ae1 commit c758ce1
Show file tree
Hide file tree
Showing 13 changed files with 136 additions and 132 deletions.
24 changes: 12 additions & 12 deletions docs/benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,25 @@

# Benchmarks

There are two sets of micro-benchmarks, one testing [round trip time](#round-trip-time-benchmarks) for both PGlite and wa-sqlite, and [another](#sqlite-benchmark-suite) based on the [SQLite speed test](https://sqlite.org/src/file?name=tool/speedtest.tcl&ci=trunk) which was ported for the [wa-sqlite benchmarks](https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html).
There are two sets of micro-benchmarks: one testing [round trip time](#round-trip-time-benchmarks) for both PGlite and wa-sqlite, and [another](#sqlite-benchmark-suite) the other based on the [SQLite speed test](https://sqlite.org/src/file?name=tool/speedtest.tcl&ci=trunk) which was ported for the [wa-sqlite benchmarks](https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html).

We also have a set of [native baseline](#native-baseline) results where we have compared native SQLite (via the Node better-sqlite3 package) to full Postgres.
We also have a set of [native baseline](#native-baseline) results comparing native SQLite (via the Node better-sqlite3 package) to full Postgres.

Comparing Postgres to SQlite is a little difficult as they are quite different databases, particularly when you then throw in the complexities of WASM. Therefore these benchmarks provide a view of performance only as a starting point to investigate the difference between the two and the improvements we can make going forward.
Comparing Postgres to SQlite is challenging, as they are quite different databases, particularly when you take into account the complexities of WASM. Therefore, these benchmarks provide a view of performance only as a starting point to investigate the difference between the two, and the improvements we can make going forward.

The other thing to consider when analysing the speed is the performance of various different VFS implementations providing persistance to both PGlite and wa-sqlite, the the performance of the underlying storage.
Another consideration when analysing the speed, is the performance of the various different VFS implementations providing persistance to both PGlite and wa-sqlite.

The key finding are:
The key findings are:

1. wa-sqlite is a little faster than PGlite when run purely in memory. This is be expected as it is a simpler database with fewer features, its designed to go fast. Having said that, PGlite is not slow, its well withing the range you would expect when [comparing native SQLite to Postgres](#native-baseline).
1. wa-sqlite is faster than PGlite when run purely in memory. This is to be expected as it's a simpler database with fewer features; it's designed to go fast. Having said that, PGlite is not slow; it's well within the range you would expect when [comparing native SQLite to Postgres](#native-baseline).

2. For single row CRUD inserts and updates, PGlite is faster then wa-sqlite. This is likely due to PGlite using the Posrgres WAL, whereas wa-sqlite is only using the SQLite rollback journal mode and not its WAL.
2. For single row CRUD inserts and updates, PGlite is faster then wa-sqlite. This is likely due to PGlite using the Postgres WAL, whereas wa-sqlite is only using the SQLite rollback journal mode and not a WAL.

3. An fsync or flush to the underlying storage can be quite slow, particularly in the browser with IndexedDB for PGlite, or OPFS for wa-sqlite. Both offer some level of "relaxed durability" that can be used to accelerate these queriers, and is likely suitable for many embedded use cases.
3. An fsync or flush to the underlying storage can be quite slow, particularly in the browser with IndexedDB for PGlite, or OPFS for wa-sqlite. Both offer some level of "relaxed durability" that can be used to accelerate these queries, and this mode is likely suitable for many embedded use cases.

We are going to continue to use these micro-benchmarks to feed back into the development of PGlite, and update them and the findings as we move forward.
We plan to continue to use these micro-benchmarks to feed back into the development of PGlite, and update them, and the findings, as we move forward.

These results below were run on a M2 Macbook Air.
These results below were run on an M2 Macbook Air.

## Round-trip-time benchmarks

Expand All @@ -63,7 +63,7 @@ Values are average ms - lower is better.

## SQLite benchmark suite

The SQLite benchmark suite, converted to web for wa-sqlite, performs a number of large queries to test the performance of the sql engin.
The SQLite benchmark suite, converted to web for wa-sqlite - it performs a number of large queries to test the performance of the sql engine.

Values are seconds to complete the test - lower is better.

Expand Down Expand Up @@ -115,7 +115,7 @@ All tests run with Node, [Better-SQLite3](https://www.npmjs.com/package/better-s

## Run the benchmarks yourself

We have a hosted version of the benchmarks runner:
We have a hosted version of the benchmark runners that you can run yourself:

- <a href="./benchmark/" target="_blank">Benchmark using the SQLite benchmark suite</a>
- <a href="./benchmark/rtt.html" target="_blank">Benchmark round-trim-time for CRUD queries</a>
Expand Down
42 changes: 21 additions & 21 deletions docs/docs/about.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,37 @@
# What is PGlite

PGlite is a WASM Postgres build packaged into a TypeScript/JavaScript client library that enables you to run Postgres in the browser, Node.js and Bun, with no need to install any other dependencies. It's under 3mb gzipped, and has support for many [Postgres extensions](../extensions/), including [pgvector](../extensions/#pgvector).
PGlite is a WASM Postgres build packaged into a TypeScript/JavaScript client library, that enables you to run Postgres in the browser, Node.js and Bun, with no need to install any other dependencies. It's under 3mb Gzipped, and has support for many [Postgres extensions](../extensions/), including [pgvector](../extensions/#pgvector).

Unlike previous "Postgres in the browser" projects, PGlite does not use a Linux virtual machine - it is simply Postgres in WASM.
Getting started with PGlite is simple: just install and import the NPM package, then create your embedded database:

It's being developed by [ElectricSQL](https://electric-sql.com/) for our use case of embedding into applications, either locally or at the edge, allowing users to sync a subset of their Postgres database.
```js
import { PGlite } from "@electric-sql/pglite";

However, there are many more use cases for PGlite beyond it's use as an embedded application databases:
const db = new PGlite();
await db.query("select 'Hello world' as message;");
// -> { rows: [ { message: "Hello world" } ] }
```

- Unit and CI testing<br>
PGlite is very fast to start and tare down, perfect for unit tests, you can a unique fresh Postgres for each test.
It can be used as an ephemeral in-memory database, or with persistence either to the file system (Node/Bun), or indexedDB (Browser).

- Local development<br>
You can use PGlite as an alternative to a full local Postgres for local development, masivly simplifyinf your development environmant.
Unlike previous "Postgres in the browser" projects, PGlite does not use a Linux virtual machine - it is simply Postgres in WASM.

- Remote development, or local web containers<br>
As PGlite is so light weight it can be easily embedded into remote containerised development environments, or in-browser [web containers](https://webcontainers.io).
It's being developed by [ElectricSQL](https://electric-sql.com/) for our use case of embedding into applications, either locally or at the edge, allowing users to sync a subset of their Postgres database.

- On-device or edge AI and RAG<br>
PGlite has full support for [pgvector](../extensions/#pgvector), enabling a local or edge retrieval augmented generation (RAG) workflow.
However, there are many more use cases for PGlite beyond its use as an embedded application database:

We are very keen to establish PGlite as an open source, and open contribution, project, working to build a community around it to develop its capabilities for all use cases.
- **Unit and CI testing**<br>
PGlite is very fast to start and tear down. It's perfect for unit tests - you can have a unique fresh Postgres for each test.

Getting started with PGlite is super easy, just install and import the NPM package, then create a your embded database:
- **Local development**<br>
You can use PGlite as an alternative to a full local Postgres for development; simplifying your development environments.

```js
import { PGlite } from "@electric-sql/pglite";
- **Remote development, or local web containers**<br>
As PGlite is so lightweight it can be easily embedded into remote containerised development environments, or in-browser [web containers](https://webcontainers.io).

const db = new PGlite();
await db.query("select 'Hello world' as message;");
// -> { rows: [ { message: "Hello world" } ] }
```
- **On-device or edge AI and RAG**<br>
PGlite has full support for [pgvector](../extensions/#pgvector), enabling a local or edge retrieval augmented generation (RAG) workflow.

It can be used as an ephemeral in-memory database, or with persistence either to the file system (Node/Bun) or indexedDB (Browser).
We are very keen to establish PGlite both as an open source, and open contribution, project, working to build a community around it, so as to develop its capabilities for all use cases.

Read more in our [getting started guide](./index.md).
46 changes: 23 additions & 23 deletions docs/docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ outline: [2, 3]
`new PGlite(dataDir: string, options: PGliteOptions)`<br/>
`new PGlite(options: PGliteOptions)`

A new pglite instance is created using the `new PGlite()` constructor.
A new PGlite instance is created using the `new PGlite()` constructor.

This is imported as:

Expand All @@ -20,15 +20,15 @@ import { PGlite } from "@electric-sql/pglite";
`await PGlite.create(dataDir: string, options: PGliteOptions)`<br />
`await PGlite.create(options: PGliteOptions)`

There is also an additional `PGlite.create()` static method that returns a Promise resolving to the new PGlite instance. There are a couple of advatanges to using the static method:
There is also a `PGlite.create()` static method that returns a promise, resolving to the new PGlite instance. There are a couple of advantages to using the static method:

- The Promise awaits the [`.waitReady`](#waitready) promise ensureing that database has fully initiated.
- When using TypeScript and extensions the returned PGlite instance will have the extensions namespace on it's type. This is not possible with the standard constructor.
- This awaits the [`.waitReady`](#waitready) promise, ensuring that the database has fully initiated.
- When using TypeScript and extensions, the returned PGlite instance will have the extensions namespace on its type. This is not possible with the standard constructor due to limitation with TypeScript.


#### `dataDir`

Path to the directory to store the Postgres database. You can provide a url scheme for various storage backends:
Path to the directory for storing the Postgres database. You can provide a url scheme for various storage backends:

- `file://` or unprefixed<br />
File system storage, available in Node and Bun.
Expand All @@ -40,23 +40,23 @@ Path to the directory to store the Postgres database. You can provide a url sche
#### `options`

- `dataDir: string`<br />
The directory to store the Postgres database when not provided as the first argument.
The directory to store the Postgres database in when not provided as the first argument.
- `debug: 1-5`<br />
the Postgres debug level. Logs are sent to the console.
- `relaxedDurability: boolean`<br />
Under relaxed durability mode PGlite will not wait for flushes to storage to complete after each query before returning results. This is particularly useful when using the indexedDB file system.
Under relaxed durability mode, PGlite will not wait for flushes to storage to complete after each query before returning results. This is particularly useful when using the indexedDB file system.
- `fs: Filesystem`<br />
The alternative to providing a dataDir with a filesystem prefix is to initiate the Filesystem yourself and provide it here. See [Filesystems](./filesystems.md)
- `loadDataDir: Blob | File`<br />
A tarball of a PGlite `datadir` to load when the database starts. This should be a tarball produced from the related [`.dumpDataDir()`](#dumpdatadir) method.
- `extensions: Extensions`<br />
An object containing the extensions you with to load.
An object containing the extensions you wish to load.

#### `options.extensions`

PGlite and Postgres extensions are loaded into a PGLite instance on start, and can include both a WASM build of a Postgres extension and/or a PGlite client plugin.

The `options.extensions` paramiter is an opbject of `namespace: extension` parings. The namespace if sued to expose the PGlite client plugin included in the extension. An example of this it the [live queries](./live-queries.md) extension.
The `options.extensions` parameter is an object of `namespace: extension` parings. The namespace if used to expose the PGlite client plugin included in the extension. An example of this is the [live queries](./live-queries.md) extension.

```ts
import { PGlite } from "@electric-sql/pglite";
Expand All @@ -65,7 +65,7 @@ import { vector } from "@electric-sql/pglite/vector";

const pg = await PGlite.create({
extensions: {
live, // Live query extension, if a PGlite client plugin
live, // Live query extension, is a PGlite client plugin
vector, // Postgres pgvector extension
},
});
Expand Down Expand Up @@ -107,7 +107,7 @@ The `query` and `exec` methods take an optional `options` objects with the follo
The returned row object type, either an object of `fieldName: value` mappings or an array of positional values. Defaults to `"object"`.
- `parsers: ParserOptions` <br />
An object of type `{[[pgType: number]: (value: string) => any;]}` mapping Postgres data type id to parser function.
For convenance the `pglite` package exports a const for most common Postgres types:
For convenience, the `pglite` package exports a constant for most common Postgres types:

```ts
import { types } from "@electric-sql/pglite";
Expand All @@ -134,7 +134,7 @@ This is useful for applying database migrations, or running multi-statement sql

Uses the *simple query* Postgres wire protocol.

Returns array of [result objects](#results-objects), one for each statement.
Returns array of [result objects](#results-objects); one for each statement.

##### Example

Expand Down Expand Up @@ -167,9 +167,9 @@ await pg.exec(`

`.transaction<T>(callback: (tx: Transaction) => Promise<T>)`

To start an interactive transaction pass a callback to the transaction method. It is passed a `Transaction` object which can be used to perform operations within the transaction.
To start an interactive transaction, pass a callback to the transaction method. It is passed a `Transaction` object which can be used to perform operations within the transaction.

The transaction will be committed when the Promise returned from your callback resolves, and automatically rolled back if the Promise is rejected.
The transaction will be committed when the promise returned from your callback resolves, and automatically rolled back if the promise is rejected.

##### `Transaction` objects

Expand Down Expand Up @@ -219,15 +219,15 @@ await pg.query("NOTIFY test, 'Hello, world!'");

`.unlisten(channel: string, callback?: (payload: string) => void): Promise<void>`

Unsubscribe from the channel. If a callback is provided it removes only that callback from the subscription, when no callback is provided is unsubscribes all callbacks for the channel.
Unsubscribe from the channel. If a callback is provided it removes only that callback from the subscription. When no callback is provided, it unsubscribes all callbacks for the channel.

### onNotification

`onNotification(callback: (channel: string, payload: string) => void): () => void`

Add an event handler for all notifications received from Postgres.

**Note:** This does not subscribe to the notification, you will have to manually subscribe with `LISTEN channel_name`.
**Note:** This does not subscribe to the notification; you will need to manually subscribe with `LISTEN channel_name`.

### offNotification

Expand All @@ -239,13 +239,13 @@ Remove an event handler for all notifications received from Postgres.

`dumpDataDir(): Promise<File | Blob>`

Dump the Postgres `datadir` to a gziped tarball.
Dump the Postgres `datadir` to a Gzipped tarball.

This can then be used in combination with the [`loadDataDir`](#options) option when starting PGlite to load a dumped database from storage.

::: tip NOTE

The datadir dump may not be compatible with other Postgres versions, it is only designed for importing back into PGlite.
The datadir dump may not be compatible with other Postgres versions; it is only designed for importing back into PGlite.

:::

Expand All @@ -271,7 +271,7 @@ Promise that resolves when the database is ready to use.

::: tip NOTE

Queries methods will wait for the `waitReady` promise to resolve if called before the database has fully initialised, and so it's not necessary to wait for it explicitly.
Query methods will wait for the `waitReady` promise to resolve if called before the database has fully initialised, and so it is not necessary to wait for it explicitly.

:::

Expand All @@ -283,7 +283,7 @@ Result objects have the following properties:
The rows retuned by the query

- `affectedRows?: number` <br />
Count of the rows affected by the query. Note this is *not* the count of rows returned, it is the number or rows in the database changed by the query.
Count of the rows affected by the query. Note, this is *not* the count of rows returned, it is the number or rows in the database changed by the query.

- `fields: { name: string; dataTypeID: number }[]`<br />
Field name and Postgres data type ID for each field returned.
Expand All @@ -300,23 +300,23 @@ The `.query<T>()` method can take a TypeScript type describing the expected shap

::: tip NOTE

These types are not validated at run time, the result only cast to the provided type
These types are not validated at run time, the result is only cast to the provided type.

:::

## /dev/blob

PGlite has support for importing and exporting via the SQL `COPY TO/FROM` command by using a virtual `/dev/blob` device.

To import a file pass the `File` or `Blob` in the query options as `blob`, and copy from the `/dev/blob` device.
To import a file, pass the `File` or `Blob` in the query options as `blob`, and copy from the `/dev/blob` device.

```ts
await pg.query("COPY my_table FROM '/dev/blob';", [], {
blob: MyBlob
})
```

To export a table or query to a file you just have to write to the `/dev/blob` device, the file will be retied as `blob` on the query results:
To export a table or query to a file, you just need to write to the `/dev/blob` device; the file will be returned as `blob` on the query results:

```ts
const ret = await pg.query("COPY my_table TO '/dev/blob';")
Expand Down
Loading

0 comments on commit c758ce1

Please sign in to comment.