Create update_monitor_maria_db.sh #12
Conversation
Script to collect info from condor queue and populate a mysql database that will be updated at regular intervals
|
@T-Britton Stuck on this error code. Seems like a permissions issue but cannot pinpoint it I wonder if it needs something like this To GRANT UPDATE privilege to user. |
There was a problem hiding this comment.
if a job finishes it may not be in condor_q. You instead need to find it in condor_history. Additionally, this looks like it updates all jobs. This will definitely not scale. during monitoring you should only query jobs which are in a non end state (e.g. Status 1 or 2 or sentinel values). Additionally, I wouldn't make monitoring do the inserts as there is a nice division of labor: during submission you do the inserts and the monitoring is a separate arm that just updates. the knock on effect is that your query for monitoring is just an update and not an insert unless it is a duplicate then update. I am actually not sure how the DB would recognize the duplicate key as there is no forced uniqueness on the other parts.
There was a problem hiding this comment.
You are right about the scaling. Too many condor queries seem to slow down the entire thing. Gotta be a little smarter about how many times condor is queried within a short period of time.
Briefly, what does this PR introduce?
Script to collect info from condor queue and populate a mysql database that will be updated at regular intervals
What kind of change does this PR introduce?
Please check if this PR fulfills the following:
Does this PR introduce breaking changes? What changes might users need to make to their code?
No
Does this PR change default behavior?
No