Skip to content

Latest commit

 

History

History
328 lines (234 loc) · 22.7 KB

README.md

File metadata and controls

328 lines (234 loc) · 22.7 KB
Raw

community

Discussion, support and common information for projects in the community.

level badge Test Donate

FAQ

What is Level?

Level is a collection of Node.js modules for creating transparent databases. A solid set of primitives enable powerful databases to be built in userland. They can be embedded or networked, persistent or transient - in short, tailored to your needs.

At the heart of Level are key-value databases that follow the characteristics of LevelDB. They support binary keys and values, batched atomic writes and bi-directional iterators that read from a snapshot in time. Entries are sorted lexicographically by keys which, when combined with ranged iterators, makes for a powerful query mechanism. Level combines idiomatic JavaScript interfaces like async iterators with Node.js interfaces like streams, events and buffers. It offers a rich set of data types through encodings and can split a database into evented sections called sublevels.

The underlying storage can be easily swapped by selecting a different database implementation, all sharing a common API and the same characteristics. Together they target a wide range of runtime environments: Node.js and Electron on Linux, Mac OS, Windows and FreeBSD, including ARM platforms like Raspberry Pi and Android, as well as Chrome, Firefox, Edge, Safari, iOS Safari and Chrome for Android.

Where do I start?

The level module is the recommended way to get started. It offers a persistent database that works in both Node.js and browsers, backed by LevelDB and IndexedDB respectively. Many alternatives are available. For example, memory-level is an in-memory database backed by a red-black tree. Visit Level/awesome to discover more modules.

What is abstract-level?

If you are new to Level, there is a quick answer: abstract-level is the new core of Level on top of which several databases are (or will be) implemented. Read on if you're already familiar with Level modules (before 2022) and have used level, levelup, abstract-leveldown, encoding-down or deferred-leveldown.

Back in 2012, levelup offered a Node.js binding for Google's LevelDB. Authored by Rod Vagg, levelup exposed the features of LevelDB in a Node.js-friendly way. It had streams, binary support, encodings... all the goodies. Later on, the binding was moved to leveldown, so that other stores could be swapped in while retaining the friendly API of levelup.

This is when "up" vs "down" naming was born, where databases followed the formula of "level = levelup + leveldown". For example, level-mem was a convenience package that bundled levelup with memdown. The abstract-leveldown module offered a lower-level abstraction for the "down" part, to encapsulate common logic between "down" stores. Many such stores were written, replacing LevelDB with IndexedDB, RocksDB, in-memory red-black trees, relational databases and more.

Around 2017, further parts were extracted from levelup and moved to single-purpose modules. This effectively introduced the concept of "layers", where an implementation of abstract-leveldown wasn't necessarily a storage for levelup but could also wrap another abstract-leveldown implementation. For example, levelup encoding logic was extracted to encoding-down. This changed the database formula to "level = levelup + encoding-down + leveldown". Or in other words: "levelup + layer + layer".

This highly modular architecture led to clean code, where each module had a single responsibility. By this time, the overall API had settled and matured, some contributors moved on to other exciting things and the primary remaining effort was maintenance. This posed new challenges. We worked on test suites, added automated browser tests, code coverage and database manifests.

Yet, releases too often required canary testing in dependents. It was hard to predict the effect of a change. In addition, documentation became fragmented and some modules actually suffered from the high modularity, having to peel off layers to customize behavior. At the same time, we could see that typical usage of a Level database still involved encodings and the other goodies that the original levelup had.

Enter abstract-level. This module merges levelup, encoding-down and abstract-leveldown into a single codebase. Instead of implementing behaviors "vertically" in layers, it is done per database method. Performance-wise abstract-level is on par with the old modules. GC pressure is lower because methods allocate less callback functions. Custom (userland) database methods also benefit from the new architecture, because they can reuse utility methods included in abstract-level rather than a layer having to detect and wrap custom methods.

Lastly, abstract-level comes with new features, some of which were not possible to implement before. Among them: Uint8Array support, built-in sublevels, atomically committing data to multiple sublevels, and reading multiple or all entries from an iterator in one call.

How do I upgrade to abstract-level?

We've put together several upgrade guides for different modules. For example, if you're currently using level@7 and no other modules (ignoring transitive dependencies) then it will suffice to read the upgrade guide of level@8.

Naming-wise, databases generally use an npm package name in the form of *-level while utilities and plugins are called level-*. This replaces the down versus up naming scheme. Similarly, while it was previously helpful for documentation to distinguish between "database" and its "underlying store", now you will mostly just encounter the term "database".

To upgrade, please consult the following table. If you use a combination of the modules listed here, each must be upgraded to its abstract-level equivalent.

Old module New module Named export 3 Upgrade guide
level <= 7 level >= 8 Level level@8
abstract-leveldown abstract-level AbstractLevel abstract-level@1
levelup n/a n/a Depends 2
level or levelup with streams level-read-stream EntryStream level-read-stream@1
leveldown classic-level ClassicLevel classic-level@1
level-mem memory-level MemoryLevel memory-level@1
memdown memory-level MemoryLevel memory-level@1
level-js browser-level BrowserLevel browser-level@1
level-rocksdb n/a (discontinued) n/a n/a
rocksdb n/a (discontinued) n/a n/a
multileveldown many-level ManyLevelGuest many-level@1
level-party rave-level RaveLevel rave-level@1
subleveldown1 n/a n/a abstract-level@1
deferred-leveldown1 n/a n/a abstract-level@1
encoding-down1 n/a n/a abstract-level@1
level-errors1 n/a n/a abstract-level@1
level-packager n/a n/a n/a
level-supports <= 2 level-supports >= 3 supports n/a
level-codec 4 level-transcoder Transcoder level-transcoder@1
level-test n/a n/a Not yet available
  1. Functionality is now included in abstract-level.
  2. If the module that you're wrapping with levelup is listed here then refer to that module's upgrade guide, else see abstract-level@1.
  3. Most new modules use named exports, for example const { ClassicLevel } = require('classic-level') instead of const leveldown = require('leveldown').
  4. Encodings that follow the level-codec interface (without level-codec as a dependency) can still be used.

Why have level-rocksdb and rocksdb been discontinued?

Maintaining these bindings to RocksDB (as an alternative to LevelDB) has been an afterthought for several years. There was no active maintainer in the Level organization who used rocksdb themselves. The level-rocksdb and rocksdb npm packages in addition have seen very few downloads compared to the main level package. LevelDB continues to be the preferred option for typical use cases of Level with good general-purpose performance.

You may find alternatives in user land.

How does Level relate to LevelDB?

Level is not affiliated with Google. Level includes a Node.js binding for Google's C++ LevelDB project. That is how Level started and where the name comes from. Since then, our ecosystem has grown to include other key-value databases besides LevelDB.

Where can I get support?

If you need help - technical, philosophical or other - feel free to open an issue in community or a more specific repository. We don't (yet) use GitHub Discussions, at least until discussions get the ability to close them.

You will generally find someone willing to help. Good questions get better and quicker answers. We do not offer paid support. All time is volunteered.

Where can I follow progress?

Most if not all activity happens on GitHub. See our project board to find out what we're working on. Any timelines there are just a rough indication of priority. We cannot guarantee that feature X or Y will actually be released on the given dates.

Subscribe to individual repositories to follow their progress. All releases are accompanied by a changelog and a GitHub Release, which gives you the option to only subscribe to new releases.

People

Collaborators

Collaborator emeriti

Contributors

Is your name missing? Send us a pull request!

API

This repository also used to hold a small amount of metadata on past and present contributors. They can be accessed from code by:

console.log(require('level-community'))

This metadata is no longer maintained and the npm package will be deprecated at some point. Contributors are instead documented in this README under People.

Contributing

Level/community is an OPEN Open Source Project. This means that:

Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.

See the Contribution Guide for more details.

Donate

Support us with a monthly donation on Open Collective and help us continue our work.

License

MIT