- [Racecar::ConsumerSet] breaking change
Racecar::ConsumerSet
's functionspoll
andbatch_pall
expect the max wait values to be given in milliseconds. The defaults were usingconfig.max_wait_time
, which is in seconds. If you do not directly useRacecar::ConsumerSet
, or always call itspoll
andbatch_poll
functions by specfiying the max wait time (the first argument), then this breaking change does not affect you. (#214)
- [Bugfix] Close RdKafka consumer in ConsumerSet#reset_current_consumer to prevent memory leak (#196)
- [Bugfix]
poll
/batch_poll
would not retry in edge cases and raise immediately. They still honor themax_wait_time
setting, but might return no messages instead and only retry on their next call. (#177)
- Bump rdkafka to 0.8.0 (#191)
- Replace
ruby-kafka
withrdkafka-ruby
as the low-level library underneath Racecar (#91). - Fix
max_wait_time
usage (#179). - Removed config option
sasl_over_ssl
. - [Racecar::Consumer] Do not pause consuming partitions on exception.
- [Racecar::Consumer]
topic
,payload
andkey
are mandadory to methodproduce
. - [Racecar::Consumer]
process_batch
retrieves an array of messages instead of batch object. - [Racecar::Consumer] Remove
offset_retention_time
. - [Racecar::Consumer] Allow providing
additional_config
for subscriptions. - [Racecar::Consumer] Provide access to
producer
andconsumer
. - [Racecar::Consumer] Enforce delivering messages with method
deliver!
. - [Racecar::Consumer] instead of raising when a partition EOF is reached, the result can be queried through
consumer.last_poll_read_partition_eof?
. - [Racecar::Config] Remove
offset_retention_time
,connect_timeout
andoffset_commit_threshold
. - [Racecar::Config] Pass config to
rdkafka-ruby
viaproducer
andconsumer
. - [Racecar::Config] Replace
max_fetch_queue_size
withmin_message_queue_size
. - [Racecar::Config] Add
synchronous_commits
to control blocking ofconsumer.commit
(defaultfalse
). - [Racecar::Config] Add
security_protocol
to control protocol between client and broker. - [Racecar::Config] SSL configuration via
ssl_ca_location
,ssl_crl_location
,ssl_keystore_location
andssl_keystore_password
. - [Racecar::Config] SASL configuration via
sasl_mechanism
,sasl_kerberos_service_name
,sasl_kerberos_principal
,sasl_kerberos_kinit_cmd
,sasl_kerberos_keytab
,sasl_kerberos_min_time_before_relogin
,sasl_username
andsasl_password
. - [Instrumentation]
produce_message.racecar
sent whenever a produced message is queued. Payload includestopic
,key
,value
andcreate_time
. - [Instrumentation]
acknowledged_message.racecar
send whenever a produced message was successfully received by Kafka. Payload includesoffset
andpartition
, but no message details. - [Instrumentation]
rdkafka-ruby
does not yet provide instrumentation rdkafka-ruby#54. - [Instrumentation] if processors define a
statistics_callback
, it will be called once every second for every subscription or producer connection. The first argument will be a Hash, for contents see librdkafka STATISTICS.md. - Add current directory to
$LOAD_PATH
only when--require
option is used (#117). - Remove manual heartbeat support, see Long-running message processing section in README.
- Rescue exceptions--then log and pass to
on_error
--at the outermost level ofexe/racecar
, so that exceptions raised outsideCli.run
are not silently discarded (#186). - When exceptions with a
cause
are logged, recursively log thecause
detail, separated by--- Caused by: ---\n
.
Unchanged from v0.5.0.
- Add support for manually sending heartbeats with
heartbeat
(#105). - Allow configuring
sasl_over_ssl
. - Add current directory to
$LOAD_PATH
only when--require
option is used (#117). - Support for
ssl_verify_hostname
in the configuration (#120)
- Allow configuring
max_bytes
andmax_fetch_queue_size
.
- Allow configuring the producer (#77).
- Add support for configuring exponential pause backoff (#76).
- Require Kafka 0.10 or higher.
- Support configuring SASL SCRAM authentication (#65).
- Change the default
max_wait_time
to 1 second. - Allow setting the
offset_retention_time
for consumers. - Allow pausing partitions indefinitely (#63).
- Allow setting the key and/or partition key when producing messages.
- Allow producing messages (alpha).
- Instrument using ActiveSupport::Notifications (#43).
- Add support for SASL.
- Use KingKonf for defining configuration variables.
- Allow setting configuration variables through the CLI.
- Make all configuration variables available over the ENV.
- Allow configuring Datadog monitoring.