Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cron start problem #1520

Open
rootcfg opened this issue Oct 15, 2024 · 20 comments
Open

Cron start problem #1520

rootcfg opened this issue Oct 15, 2024 · 20 comments

Comments

@rootcfg
Copy link

rootcfg commented Oct 15, 2024

Good day! I really ask for help! My environment is Jruby 9.4.8\Rails 7.0.8. Gem has been successfully installed and launched. All necessary tables in the database have been created and, judging by the log, are being updated. I also see my job in the cron section of the UI. The problem is that it does not start on schedule. If you launch it from the UI, it runs successfully, but cannot start on its own! What am I doing wrong?

Here all my configs:

application_job.rb

class ApplicationJob < ActiveJob::Base
self.queue_adapter = :good_job
include GoodJob::ActiveJobExtensions::Labels
retry_on ActiveRecord::Deadlocked
end

========================================

/app/jobs/search_and_notify_job.rb

cass SearchAndNotifyJob < ApplicationJob
queue_as :default

def perform
query = {
'bool': {
'must': [],
'filter': [
{
'match_all': {}
},
{
'match_phrase': {
'tag': 'log_java'
}
},
{
'match_phrase': {
'level': 'ERROR'
}
},
{
'range': {
'timestamp': {
'gte': '2024-10-09T21:00:00.000Z',
'lte': '2024-10-10T20:59:59.999Z',
'format': 'strict_date_optional_time'
}
}
}
]
}
}

@results = OpensearchClient.search(query: query, limit: 1)
results_text = @results.map { |r| "Event: #{r['_source']['throwable']}" }.join("\n")
NotificationMailer.events_report(results_text).deliver_later

Rails.logger.debug("==================DEBUG=======================")
Rails.logger.debug(results_text)
Rails.logger.debug("==================DEBUG=======================")

rescue => e
Rails.logger.error("ERROR: #{e.message}")
end
end

/config/initializers/active_job.rb

ActiveJob::Base.queue_adapter = :good_job

/config/initializers/good_job.rb

Rails.application.configure do
config.good_job.logger = Logger.new(STDOUT)
config.good_job.logger.level = Logger::DEBUG
GoodJob.logger.formatter = proc do |severity, datetime, progname, msg|
datetime = datetime.in_time_zone('Moscow') # Укажите здесь нужный часовой пояс
"#{datetime} #{severity}: #{msg}\n"
end

config.good_job.enable_cron = true
config.good_job.execution_mode = :async_all
config.good_job.active_record_parent_class = "ApplicationRecord"
config.good_job.preserve_job_records = false
config.good_job.retry_on_unhandled_error = true
config.good_job.queues = '*'
config.good_job.max_threads = 15
config.good_job.poll_interval = 10 # seconds
config.good_job.shutdown_timeout = 25 # seconds
config.good_job.cron_graceful_restart_period = 1.minutes
config.good_job.dashboard_default_locale = :en
config.good_job.on_thread_error = -> (exception) { puts "GOOD_JOB ERROR: #{exception}" }

config.good_job.cron = {

every_two_minute: {
  cron: '/2  * * *',
  class: "SearchAndNotifyJob",
}

}
end

@bensheldon
Copy link
Owner

Are you in the development environment, or is this happening in production too?

I ask because in Development, GoodJob tries not to initialize your application prematurely. That means it won't start until Active Record and Active Job is initialized.

You might see if this does it:

# config/initializers/good_job.rb
Rails.configuration.after_initialize do
  ActiveRecord::Base # load Active Record
  ActiveJob::Base # load Active Job
end

@rootcfg
Copy link
Author

rootcfg commented Oct 15, 2024

I just in development stage . I add photo my console
IMG_1770

@rootcfg
Copy link
Author

rootcfg commented Oct 15, 2024


image

@bensheldon
Copy link
Owner

Maybe your cron value is malformed? Every 2 minutes should be "*/2 * * * *"

@rootcfg
Copy link
Author

rootcfg commented Oct 15, 2024

IMG_1771 last run will newer change , only when i push entry now

@rootcfg
Copy link
Author

rootcfg commented Oct 15, 2024

Maybe your cron value is malformed? Every 2 minutes should be "*/2 * * * *"

Soorry it was my copy / paste mistake, because it closed network I typed all code. In fact schedule is correct

@rootcfg
Copy link
Author

rootcfg commented Oct 15, 2024

image

@ddd2283
Copy link

ddd2283 commented Oct 16, 2024

I also tried switching to -e production, but the behavior remained the same. Maybe you can tell me what is the trigger for starting a cron task and I might understand where the pitfalls might be

@bensheldon
Copy link
Owner

Briefly:

  • The CronManager manages the CronEntries
  • CronManager is initialized in a Capsule
  • The default Capsule is attached to GoodJob.capsule and is injected by default into the Adapter
  • The Adapter is started, which starts the capsule, in the Engine.

And to confirm, your schedule means it runs at 1252, 1254, 1256, etc. It doesn't mean "run right now, and then 2 minutes later". Just want to make sure your expectations are correct.

@ddd2283
Copy link

ddd2283 commented Oct 16, 2024

Thanks for the explanation! My expectations are that he will at least start. So far I have not been able to get it to start automatically. From the interface, if I click - run now, then it starts the job. I'll try to play with the threads. Perhaps there is something wrong in my environment, but it's confusing that there are no errors in the log.

@bensheldon
Copy link
Owner

You can check out the GoodJob demo too (which lives in /demo in this repo): https://goodjob-demo.herokuapp.com/good_job/cron_entries?locale=en

For debugging this, I’d definitely recommend running them every 1 second or something just so you don’t have to wait around for the clock.

@bensheldon
Copy link
Owner

Just so there’s no confusion:

My expectations are that he will at least start

The cron behavior is to enqueue the job with perform_later at the scheduled time; it won’t enqueue a later-scheduled job.

@rootcfg
Copy link
Author

rootcfg commented Oct 16, 2024

Yes, I definitely understand what I expect from a cron task. However, now I took a completely clean host and tried to repeat my steps. I encountered similar behavior. Is this due to the fact that I do not have any paths defined? I use ruby ​​this way -

echo '#!/bin/bash
java -jar /home/dima/jruby/jruby-complete-9.4.8.0.jar "$@"' | sudo tee /usr/local/bin/jruby
sudo chmod +x /usr/local/bin/jruby

dima@ubuntutemplate:~/jruby/my_api_app$ echo $GEM_PATH
/home/dima/jruby/jruby_gems

dima@ubuntutemplate:~/jruby/my_api_app$ echo $GEM_HOME
/home/dima/jruby/jruby_gems

dima@ubuntutemplate:~/jruby/my_api_app$ echo $PATH
/bin:/home/dima/jruby/jruby_gems/bin:/home/dima/jruby/jruby_gems/bin

@rootcfg
Copy link
Author

rootcfg commented Oct 16, 2024

I am using jruby-complete-9.4.8.0.jar from Maven repo

@rootcfg
Copy link
Author

rootcfg commented Oct 16, 2024

image

@bensheldon
Copy link
Owner

I'm not really sure where the problem lies. What happens when you press the Fast-Forward button in the dashboard? Does that fill in the "Last Run" with a job?

@ddd2283
Copy link

ddd2283 commented Oct 17, 2024

изображение

dima@ubuntutemplate:~/jruby/my_api_app$ jruby -S bundle exec good_job start
I, [2024-10-17T12:10:32.508903 #75350] INFO -- : GoodJob 4.3.0 started scheduler with queues=* max_threads=15.
I, [2024-10-17T12:10:32.831546 #75350] INFO -- : Notifier subscribed with LISTEN
D, [2024-10-17T12:11:25.365532 #75350] DEBUG -- : Notifier received payload: {"queue_name":"default","scheduled_at":"2024-10-17T12:11:24.750+03:00"}
========== MY JOB IS ALIVE!!!! ==========
I, [2024-10-17T12:11:25.943082 #75350] INFO -- : Executed GoodJob 166f0e82-f473-4273-882f-3e6814bde9eb

@ddd2283
Copy link

ddd2283 commented Oct 17, 2024

изображение
изображение
изображение

Sometimes upon restart it picks up the task itself! Perhaps the problem is somewhere in the difference in time zones or date formats?

my_api_app/config/environments/development.rb

Rails.application.configure do

config.good_job.enable_cron = true

config.time_zone = 'Moscow'
config.good_job.logging_level = :debug
config.good_job.preserve_job_records = true
config.good_job.cleanup_preserved_jobs_before_seconds_ago = 86400 # Keep tasks 24h
config.active_job.queue_adapter = :good_job

Pool settings

config.good_job.execution_mode = :external
config.good_job.max_threads = 15
config.good_job.poll_interval = 1

Cron tasks

config.good_job.cron = {
search_and_notify: {
cron: '* * * * *',
class: 'ExampleJob'
}
}

Logs

config.good_job.logger = Logger.new(STDOUT)
config.good_job.logging_level = :debug

config.cache_classes = false
config.eager_load = false
config.consider_all_requests_local = true
config.server_timing = true

if Rails.root.join("tmp/caching-dev.txt").exist?
config.cache_store = :memory_store
config.public_file_server.headers = {
"Cache-Control" => "public, max-age=#{2.days.to_i}"
}
else
config.action_controller.perform_caching = false
config.cache_store = :null_store
end

config.active_storage.service = :local
config.action_mailer.raise_delivery_errors = false
config.action_mailer.perform_caching = false
config.active_support.deprecation = :log
config.active_support.disallowed_deprecation = :raise
config.active_support.disallowed_deprecation_warnings = []
config.active_record.migration_error = :page_load
config.active_record.verbose_query_logs = true
end

@ddd2283
Copy link

ddd2283 commented Oct 18, 2024

I think the problem is in the next_at method. I think I'm not getting into the right event. My job executed but once. But why....

изображение

@rootcfg
Copy link
Author

rootcfg commented Oct 20, 2024

I also took the config from Jruby, put it in the MRI project and everything works as in the description. Could you install the version of Jruby 9.4.8.0, Rails 7.0.8.5, openjdk-21 via RVM yourself and check if it works correctly?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Inbox
Development

No branches or pull requests

3 participants