Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduler non-starts runs for some long tasks #1412

Closed
mfeit-internet2 opened this issue Mar 12, 2024 · 0 comments
Closed

Scheduler non-starts runs for some long tasks #1412

mfeit-internet2 opened this issue Mar 12, 2024 · 0 comments
Assignees
Labels

Comments

@mfeit-internet2
Copy link
Member

@cs1867 observed non-starts for 24-hour latencybg when a schedule is specified.

This appears to be an edge case in the scheduler. Slip should never be reduced to negative. Also need to see why an identical proposed run range isn't treated as common.

Relevant parts of mesh config:

{
    "groups": {
	"group_mesh_wash": {
	    "type": "mesh",
	    "addresses": [
                { "name": "foo" },
                { "name": "bar" }
            ]
	}
    },
    "tests": {
        "latency_test": {
            "type": "latencybg",
            "spec": {
	        "source": "{% address[0] %}",
	        "dest": "{% address[1] %}"
            }
	}
    },
    "schedules": {
        "schedule_P1D": {
            "repeat": "P1D",
            "sliprand": false,
            "slip": "PT3M"
        }
    },
    "tasks": {
        "latencybg_wash": {
            "group": "group_mesh_wash",
            "test": "latency_test",
            "schedule": "schedule_P1D",
            "archives": [ "http_archive" ],
            "reference": {
                "display-task-name": "Wash Loss Tests"
            }
        }
    }



}

Relevant bit of task as posted by pSConfig:

    "schedule": {
	"repeat": "P1D",
        "slip": "PT3M",
        "sliprand": false,
        "until": "2024-03-13T19:58:45Z"
    },
    "schema": 1,
    "test": {
        "spec": {
            "dest": "140.208.255.240",
	    "schema": 1,
	    "source": "137.75.71.160"
	},
	"type": "latencybg"
    },
    "tool": "powstream"

Log from scheduler:

scheduler DEBUG    scheduler-pool-3: 3: Started worker runner
scheduler DEBUG    3: Participant list is ['xxx']
scheduler DEBUG    3: Binding from None
scheduler DEBUG    3: All participants are up.
scheduler DEBUG    3: Task URLs are ['https://xxx/pscheduler/tasks/45ed0f1b-41aa-4902-bfb3-9f333968dcbb']

vvvvv  This is the interesting part:
scheduler DEBUG    3: Chopped slip to -1 day, 23:59:58
^^^^^

scheduler DEBUG    3: Possible run range {'start': '2024-03-11T19:58:55Z', 'end': '2024-03-12T19:58:55Z'}
scheduler DEBUG    3: Trying to schedule with priority None
scheduler DEBUG    3: Fetching proposals from https://xxx/pscheduler/tasks/45ed0f1b-41aa-4902-bfb3-9f333968dcbb/runtimes
scheduler DEBUG    3: Ranges: [R(2024-03-11 19:58:55+00:00..2024-03-12 19:58:55+00:00)]
scheduler DEBUG    3: Done fetching time ranges

vvvvv Also interesting because the one proposed range matches the possible range exactly.
scheduler DEBUG    3: Ranges in common: []
^^^^^
scheduler DEBUG    3: Unable: No times available for this run.
scheduler INFO     3: Posting non-starting run at 2024-03-11T19:58:50Z for task 45ed0f1b-41aa-4902-bfb3-9f333968dcbb: No times available for this run.
scheduler DEBUG    3: Thread finished
@mfeit-internet2 mfeit-internet2 self-assigned this Mar 12, 2024
@github-project-automation github-project-automation bot moved this to Ready in perfSONAR Mar 12, 2024
@mfeit-internet2 mfeit-internet2 changed the title Scheduler non-atarts runs for some long tasks Scheduler non-starts runs for some long tasks Mar 13, 2024
@github-project-automation github-project-automation bot moved this from Ready to Done in perfSONAR Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

No branches or pull requests

1 participant