Skip to content

Commit

Permalink
First release of River Ruby bindings
Browse files Browse the repository at this point in the history
A first push of Ruby bindings for River. Meant to be used in conjunction
with a driver like `riverqueue-sequel` [1] to provide an insert-only
client for River. See the README for details on usage.

Overall, I'm happy at how close I was able to keep the API to the Go
version. A lot of syntax in Go just isn't needed due to the more dynamic
and implicit nature of Ruby, but the parts that came through are quite
close. e.g. We have a job args concept, along with `InsertOpts` that can
be added to both jobs and at insert time, just like Go.

Purposely not implemented on this first push (I'll follow up with these
later on):

* Unique jobs.
* Batch insert.

[1] https://github.com/riverqueue/riverqueue-ruby-sequel
  • Loading branch information
brandur committed Nov 20, 2023
1 parent 7437932 commit 9f55cc7
Show file tree
Hide file tree
Showing 14 changed files with 554 additions and 16 deletions.
72 changes: 72 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: CI

env:
# Database to connect to that can create other databases with `CREATE DATABASE`.
ADMIN_DATABASE_URL: postgres://postgres:postgres@localhost:5432

# Just a common place for steps to put binaries they need and which is added
# to GITHUB_PATH/PATH.
BIN_PATH: /home/runner/bin

# A suitable URL for a test database.
TEST_DATABASE_URL: postgres://postgres:[email protected]:5432/river_ruby_test?sslmode=disable

on:
- push

jobs:
lint:
runs-on: ubuntu-latest
timeout-minutes: 3

spec:
runs-on: ubuntu-latest
timeout-minutes: 3

steps:
- name: Checkout
uses: actions/checkout@v4

- Install Ruby + `bundle install`
uses: ruby/setup-ruby@v1
with:
ruby-version: "head"
bundler-cache: true # runs 'bundle install' and caches installed gems automatically

- name: Standard Ruby
run: bundle exec standardrb

services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 2s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432

steps:
- name: Checkout
uses: actions/checkout@v4

- Install Ruby + `bundle install`
uses: ruby/setup-ruby@v1
with:
ruby-version: "head"
bundler-cache: true # runs 'bundle install' and caches installed gems automatically

- name: Create database
run: psql --echo-errors --quiet -c '\timing off' -c "CREATE DATABASE river_ruby_test;" ${ADMIN_DATABASE_URL}

- name: Install River CLI
run: go install github.com/riverqueue/river/cmd/river@latest

- name: river migrate-up
run: ./river migrate-up --database-url $DATABASE_URL

- name: Rspec
run: bundle exec rspec
12 changes: 12 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
source "https://rubygems.org"

gemspec

group :development, :test do
gem "standard"
end

group :test do
gem "rspec-core"
gem "rspec-expectations"
end
69 changes: 69 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
PATH
remote: .
specs:
riverqueue (0.0.1)

GEM
remote: https://rubygems.org/
specs:
ast (2.4.2)
diff-lcs (1.5.0)
json (2.6.3)
language_server-protocol (3.17.0.3)
lint_roller (1.1.0)
parallel (1.23.0)
parser (3.2.2.4)
ast (~> 2.4.1)
racc
racc (1.7.3)
rainbow (3.1.1)
regexp_parser (2.8.2)
rexml (3.2.6)
rspec-core (3.12.2)
rspec-support (~> 3.12.0)
rspec-expectations (3.12.3)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.12.0)
rspec-support (3.12.1)
rubocop (1.57.2)
json (~> 2.3)
language_server-protocol (>= 3.17.0)
parallel (~> 1.10)
parser (>= 3.2.2.4)
rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 1.8, < 3.0)
rexml (>= 3.2.5, < 4.0)
rubocop-ast (>= 1.28.1, < 2.0)
ruby-progressbar (~> 1.7)
unicode-display_width (>= 2.4.0, < 3.0)
rubocop-ast (1.30.0)
parser (>= 3.2.1.0)
rubocop-performance (1.19.1)
rubocop (>= 1.7.0, < 2.0)
rubocop-ast (>= 0.4.0)
ruby-progressbar (1.13.0)
standard (1.32.0)
language_server-protocol (~> 3.17.0.2)
lint_roller (~> 1.0)
rubocop (~> 1.57.2)
standard-custom (~> 1.0.0)
standard-performance (~> 1.2)
standard-custom (1.0.2)
lint_roller (~> 1.0)
rubocop (~> 1.50)
standard-performance (1.2.1)
lint_roller (~> 1.1)
rubocop-performance (~> 1.19.1)
unicode-display_width (2.5.0)

PLATFORMS
arm64-darwin-22

DEPENDENCIES
riverqueue!
rspec-core
rspec-expectations
standard

BUNDLED WITH
2.4.20
8 changes: 0 additions & 8 deletions README.md

This file was deleted.

21 changes: 21 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# River client for Ruby [![Build Status](https://github.com/riverqueue/riverqueue-ruby/workflows/CI/badge.svg)](https://github.com/riverqueue/riverqueue-ruby/actions)

An insert-only Ruby client for [River](https://github.com/riverqueue/river) packaged in the [`riverqueue` gem](https://rubygems.org/gems/riverqueue).

`Gemfile` should contain the core gem and a driver like [`rubyqueue-sequel`](https://github.com/riverqueue/riverqueue-ruby-sequel):

``` yaml
gem "riverqueue"
gem "riverqueue-sequel"
```

Initialize a client with:

```ruby
DB = Sequel.connect("postgres://...")
client = River::Client.new(River::Driver::Sequel.new(DB))
```

## Development

See [development](./development.md).
39 changes: 39 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# riverqueue-ruby development

## Install dependencies

```shell
$ bundle install
```
## Run tests

Create a test database and migrate with River's CLI:

```shell
$ go install github.com/riverqueue/river/cmd/river
$ createdb riverqueue_ruby_test
$ river migrate-up --database-url "postgres://localhost/riverqueue_ruby_test"
```

Run all specs:

```shell
$ bundle exec rspec spec
```

## Run lint

```shell
$ standardrb --fix
```

## Publish a new gem

```shell
git checkout master && git pull --rebase
VERSION=v0.0.x
gem build riverqueue.gemspec
gem push riverqueue-$VERSION.gem
git tag $VERSION
git push --tags
```
56 changes: 56 additions & 0 deletions lib/client.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
module River
MAX_ATTEMPTS_DEFAULT = 25
PRIORITY_DEFAULT = 1
QUEUE_DEFAULT = "default"

# Provides a client for River that inserts jobs. Unlike the Go version of the
# River client, this one can insert jobs only. Jobs can only be worked from Go
# code, so job arg kinds and JSON encoding details must be shared between Ruby
# and Go code.
#
# Used in conjunction with a River driver like:
#
# DB = Sequel.connect(...)
# client = River::Client.new(River::Driver::Sequel.new(DB))
#
# River drivers are found in separate gems like `riverqueue-sequel` to help
# minimize transient dependencies.
class Client
def initialize(driver)
@driver = driver
end

# Inserts a new job for work given a job args implementation and insertion
# options (which may be omitted).
#
# Job arg implementations are expected to respond to:
#
# * `#kind`: A string that uniquely identifies the job in the database.
# * `#to_json`: Encodes the args to JSON for persistence in the database.
# Must match encoding an args struct on the Go side to be workable.
#
# They may also respond to `#insert_opts` which is expected to return an
# `InsertOpts` that contains options that will apply to all jobs of this
# kind. Insertion options provided as an argument to `#insert` override
# those returned by job args.
def insert(args, insert_opts: InsertOpts.new)
raise "args should respond to `#kind`" if !args.respond_to?(:kind)
raise "args should respond to `#to_json`" if !args.respond_to?(:to_json)

args_insert_opts = args.respond_to?(:insert_opts) ? args.insert_opts : InsertOpts.new

scheduled_at = insert_opts.scheduled_at || args_insert_opts.scheduled_at

@driver.insert(Internal::JobInsertParams.new(
encoded_args: args.to_json,
kind: args.kind,
max_attempts: insert_opts.max_attempts || args_insert_opts.max_attempts || MAX_ATTEMPTS_DEFAULT,
priority: insert_opts.priority || args_insert_opts.priority || PRIORITY_DEFAULT,
queue: insert_opts.queue || args_insert_opts.queue || QUEUE_DEFAULT,
scheduled_at: scheduled_at, # database default to now
state: scheduled_at ? JOB_STATE_SCHEDULED : JOB_STATE_AVAILABLE,
tags: insert_opts.tags || args_insert_opts.tags
))
end
end
end
62 changes: 62 additions & 0 deletions lib/insert_opts.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
module River
class InsertOpts
# MaxAttempts is the maximum number of total attempts (including both the
# original run and all retries) before a job is abandoned and set as
# discarded.
attr_accessor :max_attempts

# Priority is the priority of the job, with 1 being the highest priority and
# 4 being the lowest. When fetching available jobs to work, the highest
# priority jobs will always be fetched before any lower priority jobs are
# fetched. Note that if your workers are swamped with more high-priority jobs
# then they can handle, lower priority jobs may not be fetched.
#
# Defaults to PRIORITY_DEFAULT.
attr_accessor :priority

# Queue is the name of the job queue in which to insert the job.
#
# Defaults to QUEUE_DEFAULT.
attr_accessor :queue

# ScheduledAt is a time in future at which to schedule the job (i.e. in
# cases where it shouldn't be run immediately). The job is guaranteed not
# to run before this time, but may run slightly after depending on the
# number of other scheduled jobs and how busy the queue is.
#
# Use of this option generally only makes sense when passing options into
# Insert rather than when a job args is returning `#insert_opts`, however,
# it will work in both cases.
attr_accessor :scheduled_at

# Tags are an arbitrary list of keywords to add to the job. They have no
# functional behavior and are meant entirely as a user-specified construct
# to help group and categorize jobs.
#
# If tags are specified from both a job args override and from options on
# Insert, the latter takes precedence. Tags are not merged.
attr_accessor :tags

# UniqueOpts returns options relating to job uniqueness. An empty struct
# avoids setting any worker-level unique options.
#
# TODO: Implement.
attr_accessor :unique_opts

def initialize(
max_attempts: nil,
priority: nil,
queue: nil,
scheduled_at: nil,
tags: nil,
unique_opts: nil
)
self.max_attempts = max_attempts
self.priority = priority
self.queue = queue
self.scheduled_at = scheduled_at
self.tags = tags
self.unique_opts = unique_opts
end
end
end
44 changes: 44 additions & 0 deletions lib/internal.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
module River
module Internal
# Insert parameters for a job. This is sent to underlying drivers and is meant
# for internal use only. Its interface is subject to change.
class JobInsertParams
attr_accessor :encoded_args
attr_accessor :kind
attr_accessor :max_attempts
attr_accessor :priority
attr_accessor :queue
attr_accessor :scheduled_at
attr_accessor :state
attr_accessor :tags

# TODO(brandur): Get these supported.
# attr_accessor :unique
# attr_accessor :unique_by_args
# attr_accessor :unique_by_period
# attr_accessor :unique_by_queue
# attr_accessor :unique_by_state

def initialize(
encoded_args: nil,
kind: nil,
max_attempts: nil,
priority: nil,
queue: nil,
scheduled_at: nil,
state: nil,
tags: nil
)
self.encoded_args = encoded_args
self.kind = kind
self.max_attempts = max_attempts
self.priority = priority
self.queue = queue
self.scheduled_at = scheduled_at
self.state = state
self.tags = tags
end
end
end
private_constant :Internal
end
Loading

0 comments on commit 9f55cc7

Please sign in to comment.