-
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
5 changed files
with
125 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,78 @@ | ||
# Rekalogika Collections | ||
# rekalogika/collections | ||
|
||
TBD | ||
Pragmatic, opinionated enhancements to Doctrine's Collections library. Improves | ||
the use of Doctrine Collections in large datasets, and other common problems. | ||
|
||
Full documentation: [rekalogika.dev/collections](https://rekalogika.dev/collections) | ||
|
||
## Background | ||
|
||
We work with huge datasets that are managed by Doctrine ORM, and have complex | ||
business rules. These come with these challenges: | ||
|
||
* With standard Doctrine, it seems it is too easy to introduce bugs that will | ||
accidentally load the entire dataset into memory and cause out-of-memory | ||
errors. And these sorts of errors will usually only show up in production, but | ||
never in the development environment. | ||
|
||
* Iterating over large datasets with the correct method is difficult and | ||
cumbersome. You usually need to devise custom solutions for each use case. | ||
|
||
* Counting the number of records is very slow. Sometimes we can do away with the | ||
count, sometimes it is a must, and we need to work around the problem. | ||
|
||
Other, non-performance issues include: | ||
|
||
* Doctrine's `Selectable` appears to be a prevalent source of abstraction leak. | ||
Coders tend to litter the codebase with internal-dependent `Criteria` objects, | ||
and updating the entity can potentially become a nightmare. No static analysis | ||
tool can currently detect this problem. In fact, some exacerbate the problem | ||
by assuming a `Collection` must also be a `Selectable`. | ||
|
||
Previously, we created `rekalogika/doctrine-collections-decorator` to solve | ||
these problems. However, it is still too cumbersome because we need to approach | ||
the problem one at a time. We need a more comprehensive solution that applies | ||
to most, if not all, of our use cases. | ||
|
||
## Components | ||
|
||
* Decorator classes that enhance any Doctrine Collections classes. | ||
* Query-backed collections. Turns a `QueryBuilder` into a lazy-loading | ||
collection. | ||
* An alternative implementation of the repository pattern that implements | ||
`Collection`. | ||
* Modifications to `ArrayCollection` that does `matching()` against the private | ||
properties directly, to reproduce the same behavior of `PersistentCollection`. | ||
|
||
## Features | ||
|
||
* Safeguards against potential out-of-memory situations. Throws an exception | ||
before it hits harder-to-debug out-of-memory situation. | ||
* Pluggable counting strategies. Work around long counting times using your own | ||
counting strategies. | ||
* Full versions of the collection classes that offer full compatibility with the | ||
original Doctrine Collection. And the minimal flavors that only expose the | ||
safe methods. | ||
* Built in keyset pagination using | ||
[`rekalogika/rekapager`](https://rekalogika.dev/rekapager) library. Iterate | ||
over collections of any size without loading them all into memory. And without | ||
having to create ad-hoc queries every time you need to achieve that. | ||
* Keyset pagination can also be used to create pagination for user interfaces | ||
and API outputs. | ||
* Encourages you to create expressive, higher-level methods to provide the same | ||
functionality as the `Selectable` interface, but without exposing the inner | ||
workings of the class. | ||
|
||
## Documentation | ||
|
||
TBD | ||
[rekalogika.dev/collections](https://rekalogika.dev/collections) | ||
|
||
## License | ||
|
||
MIT | ||
MIT | ||
|
||
## Contributing | ||
|
||
This library consists of multiple repositories split from a monorepo. Be sure to | ||
submit issues and pull requests to the | ||
[rekalogika/collections](https://github.com/rekalogika/collections) monorepo. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,18 @@ | ||
# Rekalogika Collections Domain | ||
# rekalogika/collections-common | ||
|
||
TBD | ||
Common library required by other components of the `rekalogika/collections` | ||
library. | ||
|
||
## Documentation | ||
|
||
TBD | ||
[rekalogika.dev/collections](https://rekalogika.dev/collections) | ||
|
||
## License | ||
|
||
MIT | ||
MIT | ||
|
||
## Contributing | ||
|
||
This library consists of multiple repositories split from a monorepo. Be sure to | ||
submit issues and pull requests to the | ||
[rekalogika/collections](https://github.com/rekalogika/collections) monorepo. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,28 @@ | ||
# Rekalogika Collections Domain | ||
# rekalogika/collections-domain | ||
|
||
TBD | ||
Transforms a Doctrine `Collection` object into our `Recollection` object, which | ||
extends `Collection` itself but also extends `PageableInterface` from our | ||
`rekalogika/rekapager` library. | ||
|
||
The features include: | ||
|
||
* Safeguards against potential out-of-memory situations. | ||
* Pluggable counting strategies. | ||
* Keyset pagination for batch processing and user interfaces. | ||
|
||
The classes also available in the minimal flavor, which only exposes the safe | ||
methods, those which won't trigger full load of an extra-lazy collection. | ||
|
||
## Documentation | ||
|
||
TBD | ||
[rekalogika.dev/collections](https://rekalogika.dev/collections) | ||
|
||
## License | ||
|
||
MIT | ||
MIT | ||
|
||
## Contributing | ||
|
||
This library consists of multiple repositories split from a monorepo. Be sure to | ||
submit issues and pull requests to the | ||
[rekalogika/collections](https://github.com/rekalogika/collections) monorepo. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,25 @@ | ||
# Rekalogika Collections Domain | ||
# rekalogika/collections-orm | ||
|
||
TBD | ||
A collection class using Doctrine ORM `QueryBuilder` as the data source. Unlike | ||
doing the query in the traditional way, this class allows lazy loading. You can | ||
safely pass the object around, and it will only execute the query when you start | ||
getting items from it. | ||
|
||
The class also implements the `PageableInterface` from the | ||
[`rekalogika/rekapager`](https://rekalogika.dev/rekapager) library. This allows | ||
you to iterate over the collection without loading all the items into memory. | ||
And also useful for creating paginated user interfaces and API outputs. | ||
|
||
## Documentation | ||
|
||
TBD | ||
[rekalogika.dev/collections](https://rekalogika.dev/collections) | ||
|
||
## License | ||
|
||
MIT | ||
MIT | ||
|
||
## Contributing | ||
|
||
This library consists of multiple repositories split from a monorepo. Be sure to | ||
submit issues and pull requests to the | ||
[rekalogika/collections](https://github.com/rekalogika/collections) monorepo. |