Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Batch job processing #470

Closed
wants to merge 20 commits into from
Closed

Batch job processing #470

wants to merge 20 commits into from

Commits on Jun 6, 2024

  1. Hoist getJob to workerPool

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    fa50044 View commit details
    Browse the repository at this point in the history

  2. Switch to using poolId rather than workerId for locking/unlocking jobs

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    57eb29d View commit details
    Browse the repository at this point in the history

  3. Fix test

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    a771d63 View commit details
    Browse the repository at this point in the history

  4. Add release notes

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    5877677 View commit details
    Browse the repository at this point in the history

  5. Update docs and tests to refer to pool rather than worker for locked_at

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    6d7a9a2 View commit details
    Browse the repository at this point in the history

  6. Switch failJobs to using poolId too

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    f481a42 View commit details
    Browse the repository at this point in the history

  7. Start work on batch fetching jobs

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    88de944 View commit details
    Browse the repository at this point in the history

  8. Batched job fetching with watermark

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    454aed5 View commit details
    Browse the repository at this point in the history

  9. Fix getJob call to reflect changes in #469

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    ec38935 View commit details
    Browse the repository at this point in the history

  10. Hoist completeJob and failJob

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    2b60028 View commit details
    Browse the repository at this point in the history

  11. Refactor failJob/completeJob in preparation for batching

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    2695b50 View commit details
    Browse the repository at this point in the history

  12. Stub batch function

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    63bbceb View commit details
    Browse the repository at this point in the history

  13. Implement batching function

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    daa110e View commit details
    Browse the repository at this point in the history

  14. Lint fix

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    454d220 View commit details
    Browse the repository at this point in the history

  15. Tweak graceful/forceful shutdown handover

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    1e818db View commit details
    Browse the repository at this point in the history

  16. Evaluate envvar just once up top

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    22e8b19 View commit details
    Browse the repository at this point in the history

  17. Release batches before releasing worker pool

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    9d9207e View commit details
    Browse the repository at this point in the history

  18. Tweak perfTest for more logging

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    bf103b9 View commit details
    Browse the repository at this point in the history

  19. Fairer startup time calc

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    8fb0174 View commit details
    Browse the repository at this point in the history

  20. Refactor

    @benjie
    benjie committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    f46ec8d View commit details
    Browse the repository at this point in the history