Skip to content

Releases: Shopify/job-iteration

v1.8.0

10 Dec 14:16
b0cf2ea
Compare
Choose a tag to compare

Changes

  • 513 Deprecate returning enumerators from build_enumerator that are not wrapped with enumerator_builder.wrap. The built-in enumerator builders now always wrap.

Features

  • 340 Add cursor.iteration instrumentation for the query to fetch the next batch of records for the Active Record cursor.
  • 523 Add interruption adapter for aws-activejob-sqs.

Bug fixes

  • 515 Fix size of array enumerators.

Full Diff: v1.7.0...v1.8.0

v1.7.0

11 Oct 14:15
644613c
Compare
Choose a tag to compare

Features

  • 509 - Added CSV batching functionality to EnumeratorBuilder with build_csv_enumerator_on_batches method and csv_on_batches alias.
  • 512 - Added support for custom timezones on ActiveRecordEnumerator and ActiveRecordBatchEnumerator. This allows for using cursors with datetime columns where ActiveRecord.default_timezone is set to :local and Active Record is not using the same timezone as the database.

Full Diff: v1.6.0...v1.7.0

v1.6.0

25 Sep 13:32
f8081e7
Compare
Choose a tag to compare

Features

Full Diff: v1.5.1...v1.6.0

v1.5.1

29 May 23:47
10787b9
Compare
Choose a tag to compare

v1.5.0

29 May 20:15
2897431
Compare
Choose a tag to compare

Changes

  • 437 - Use minimum between per-class job_iteration_max_job_runtime and JobIteration.max_job_runtime, instead of enforcing only setting decreasing values.
    Because it is possible to change the global or parent values after setting the value on a class, it is not possible to truly enforce the decreasing value constraint. Instead, we now use the minimum between the global value and per-class value. This is considered a non-breaking change, as it should not break any existing code, it only removes the constraint on new classes.
  • 443 - Use Sidekiq :quit callback to detect graceful shutdown. This makes job-iteration compatible with Sidekiq run in embedded mode.
  • 445 - Add the around_iterate callback, which runs around each call of each_iteration. This adds extensibility to build some generic handlers, such as metrics collection and logging.
  • 450 - Infer which interruption adapter to use from the queue adapter of the job. This deprecates setting JobIteration.interruption_adapter = <callable>, in favor of JobIteration.register_interruption_adapter(<queue adapter name>, <callable>). JobIteration.interruption_adapter will be removed in a future release.

Bug fixes

  • 437 - Defer reading JobIteration.max_job_runtime until runtime, instead of closing around the value at the time of job definition.
  • 431 - Use #id_value instead of send(:id)
    when generating position for cursor based on :id column (Rails 7.1 and above, where composite
    primary models are now supported). This ensures we grab the value of the id column, rather than a
    potentially composite primary key value.
  • 456 - Use Arel to generate SQL that's type compatible for the
    cursor pagination conditionals in ActiveRecord cursor. Previously, the cursor would coerce numeric ids to a string value
    (e.g.: ... AND id > '1')

Full Diff: v1.4.1...v1.5.0

v1.4.1

05 Sep 09:57
0bfc557
Compare
Choose a tag to compare

Bug fixes

  • 427 - Use the Rails application logger. Changes from 338 resulted in logging to the original value of ActiveJob.logger, not the one configured by the Rails application.

Full diff: v1.4.0...v1.4.1

v1.4.0

23 Aug 20:15
7e6c385
Compare
Choose a tag to compare

Changes

  • 338 - All logs are now ActiveSupport::Notifications events and logged using ActiveSupport::LogSubscriber to allow customization. Events now always include the cursor_position tag.
  • 418 - Return nil from Iteration#perform, to signal not to rely on return value.

Features

  • 240 - Allow setting inheritable per-job job_iteration_max_job_runtime
  • 310 - Support nested iteration
  • 341 - Add JobIteration.default_retry_backoff, which sets a default delay when jobs are re-enqueued after being interrupted. Defaults to nil, meaning no delay, which matches the current behaviour.
  • 365 - Support composite primary key as a cursor

Bug fixes

  • 289 - Fix uninitialized constant error when raising ConditionNotSupportedError from ActiveRecordBatchEnumerator
  • 346 - Include failed jobs in total_time
  • 417 - Ensure that numerical values are deserialized as such and not as strings.

Full Diff: v1.3.6...v1.4.0