Releases: Shopify/job-iteration
Releases · Shopify/job-iteration
v1.8.0
Changes
- 513 Deprecate returning enumerators from
build_enumerator
that are not wrapped withenumerator_builder.wrap
. The built-in enumerator builders now always wrap.
Features
- 340 Add
cursor.iteration
instrumentation for the query to fetch the next batch of records for the Active Record cursor. - 523 Add interruption adapter for aws-activejob-sqs.
Bug fixes
- 515 Fix size of array enumerators.
Full Diff: v1.7.0...v1.8.0
v1.7.0
Features
- 509 - Added CSV batching functionality to
EnumeratorBuilder
withbuild_csv_enumerator_on_batches
method andcsv_on_batches
alias. - 512 - Added support for custom timezones on
ActiveRecordEnumerator
andActiveRecordBatchEnumerator
. This allows for using cursors with datetime columns whereActiveRecord.default_timezone
is set to:local
and Active Record is not using the same timezone as the database.
Full Diff: v1.6.0...v1.7.0
v1.6.0
Features
- 464 - Add interruption adapter for GoodJob.
- 505 - Add interruption adapter for Solid Queue.
Full Diff: v1.5.1...v1.6.0
v1.5.1
v1.5.0
Changes
- 437 - Use minimum between per-class
job_iteration_max_job_runtime
andJobIteration.max_job_runtime
, instead of enforcing only setting decreasing values.
Because it is possible to change the global or parent values after setting the value on a class, it is not possible to truly enforce the decreasing value constraint. Instead, we now use the minimum between the global value and per-class value. This is considered a non-breaking change, as it should not break any existing code, it only removes the constraint on new classes. - 443 - Use Sidekiq
:quit
callback to detect graceful shutdown. This makes job-iteration compatible with Sidekiq run in embedded mode. - 445 - Add the
around_iterate
callback, which runs around each call ofeach_iteration
. This adds extensibility to build some generic handlers, such as metrics collection and logging. - 450 - Infer which interruption adapter to use from the queue adapter of the job. This deprecates setting
JobIteration.interruption_adapter = <callable>
, in favor ofJobIteration.register_interruption_adapter(<queue adapter name>, <callable>)
.JobIteration.interruption_adapter
will be removed in a future release.
Bug fixes
- 437 - Defer reading
JobIteration.max_job_runtime
until runtime, instead of closing around the value at the time of job definition. - 431 - Use
#id_value
instead ofsend(:id)
when generating position for cursor based on:id
column (Rails 7.1 and above, where composite
primary models are now supported). This ensures we grab the value of the id column, rather than a
potentially composite primary key value. - 456 - Use Arel to generate SQL that's type compatible for the
cursor pagination conditionals in ActiveRecord cursor. Previously, the cursor would coerce numeric ids to a string value
(e.g.:... AND id > '1'
)
Full Diff: v1.4.1...v1.5.0
v1.4.1
Bug fixes
- 427 - Use the Rails application logger. Changes from 338 resulted in logging to the original value of ActiveJob.logger, not the one configured by the Rails application.
Full diff: v1.4.0...v1.4.1
v1.4.0
Changes
- 338 - All logs are now
ActiveSupport::Notifications
events and logged usingActiveSupport::LogSubscriber
to allow customization. Events now always include thecursor_position
tag. - 418 - Return
nil
fromIteration#perform
, to signal not to rely on return value.
Features
- 240 - Allow setting inheritable per-job
job_iteration_max_job_runtime
- 310 - Support nested iteration
- 341 - Add
JobIteration.default_retry_backoff
, which sets a default delay when jobs are re-enqueued after being interrupted. Defaults tonil
, meaning no delay, which matches the current behaviour. - 365 - Support composite primary key as a cursor
Bug fixes
- 289 - Fix uninitialized constant error when raising
ConditionNotSupportedError
fromActiveRecordBatchEnumerator
- 346 - Include failed jobs in
total_time
- 417 - Ensure that numerical values are deserialized as such and not as strings.
Full Diff: v1.3.6...v1.4.0