Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CosmosDBMongoTrigger should support more options (as per CosmosTrigger) #11

Open
asos-martinsmith opened this issue Feb 12, 2024 · 0 comments

Comments

@asos-martinsmith
Copy link

asos-martinsmith commented Feb 12, 2024

I would expect to see

  • StartFromBeginning/StartFromTime attributes
  • LeaseContainerPrefix
  • PreferredLocations

When using function apps with the CosmosTrigger I regularly have a requirement to just process changes from some specific time.

The GetChangeStreamTokens command supports a startAtOperationTime parameter so the first two should be simple to implement.

Ability to specify a StartFromTime is useful for numerous cases - one such use case is restoring a database to a point in time and then using the change feed process to upsert changes since the point in time of the restore in order to get an up-to-date copy.

The ability to set a new LeaseContainerPrefix in conjunction with StartFromTime allows re-processing to start again from a specific point and is useful for recovering after errors. For the Mongo API case this may also be useful for regenerating leases if (due to partition splits or merges) the leases calculated at startup get very out of synch with the physical partitions that exist.

Also it appears as though the leases collection can't currently be used to hold leases from multiple collections.

IMO the implementation for the NoSQL API change feed processor should have been followed more closely (in that the documents need a prefix that uniquely identifies the account/database/collection/lease prefix it relates to rather than just catering for one).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant