Skip to content

[FEAT] Propagate configs to Ray remote functions #3291

[FEAT] Propagate configs to Ray remote functions

[FEAT] Propagate configs to Ray remote functions #3291

Triggered via pull request December 8, 2023 01:00
Status Success
Total duration 52s
Artifacts

release-drafter.yml

on: pull_request
update_release_draft
6s
update_release_draft
Fit to window
Zoom out
Zoom in

Annotations

2 errors
update_release_draft
Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"} { name: 'HttpError', id: '7135962636', status: 422, response: { url: 'https://api.github.com/repos/Eventual-Inc/Daft/releases/132770221', status: 422, headers: { 'access-control-allow-origin': '*', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', connection: 'close', 'content-length': '195', 'content-security-policy': "default-src 'none'", 'content-type': 'application/json; charset=utf-8', date: 'Fri, 08 Dec 2023 01:00:27 GMT', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', server: 'GitHub.com', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', vary: 'Accept-Encoding, Accept, X-Requested-With', 'x-accepted-github-permissions': 'contents=write', 'x-content-type-options': 'nosniff', 'x-frame-options': 'deny', 'x-github-api-version-selected': '2022-11-28', 'x-github-media-type': 'github.v3; format=json', 'x-github-request-id': 'AD87:69AA:476496:94E7A6:65726AAB', 'x-ratelimit-limit': '1000', 'x-ratelimit-remaining': '962', 'x-ratelimit-reset': '1702000249', 'x-ratelimit-resource': 'core', 'x-ratelimit-used': '38', 'x-xss-protection': '0' }, data: { message: 'Validation Failed', errors: [ { resource: 'Release', code: 'invalid', field: 'target_commitish' } ], documentation_url: 'https://docs.github.com/rest/releases/releases#update-a-release' } }, request: { method: 'PATCH', url: 'https://api.github.com/repos/Eventual-Inc/Daft/releases/132770221', headers: { accept: 'application/vnd.github.v3+json', 'user-agent': 'probot/12.2.5 octokit-core.js/3.5.1 Node.js/16.20.2 (linux; x64)', authorization: 'token [REDACTED]', 'content-type': 'application/json; charset=utf-8' }, body: '{"body":"## Changes\\n\\n## ✨ New Features\\n\\n- [FEAT] [JSON Reader] Add native streaming + parallel JSON reader. @clarkzinzow (#1679)\\n\\n## 🚀 Performance Improvements\\n\\n- [PERF] Enable Predicates in Parquet Reader @samster25 (#1702)\\n\\n## 📖 Documentation\\n\\n- [DOCS] Add notebooks used for pydata global 2023 presentation @jaychia (#1703)\\n","draft":true,"prerelease":false,"make_latest":"true","name":"v0.2.7","tag_name":"v0.2.7","target_commitish":"refs/pull/1707/merge"}', request: {} }, event: { id: '7135962636', name: 'pull_request', payload: { action: 'edited', changes: { body: { from: '## General Changes\r\n' + '\r\n' + '1. Adds serde and pickling `__reduce__` impl for `PyDaftConfig` so it can be sent over the wire in Ray\r\n' + '2. Parametrizes our runners (both PyRunner and RayRunner) with a `DaftConfig` object upon initialization. We thus also have to make it illegal from the user-API to set the configs once the runners have been instantiated (i.e. when any creation/execution of dataframes has occurred)\r\n' + '3. I also consolidated our `_RUNNER` and `_DAFT_CONTEXT` global singletons in the `daft.context` module -- the runner now resides inside the `DaftContext` object\r\n' + '\r\n' + '## RayRunner changes\r\n' + '\r\n' + '1. For every call to `@ray.remote`, ensure that we accept a daft_config as the first arg and call `set_config` as the first action\r\n' + '4. When the `RayRunner` is initialized, we call a `ray.put` to put the DaftConfig into the cluster store that objectref. This objectref is also passed around to various other datastructures that may potentially call `ray.remote` functio
update_release_draft
HttpError: Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"} at /home/runner/work/_actions/release-drafter/release-drafter/v5/dist/index.js:8462:21 at processTicksAndRejections (node:internal/process/task_queues:96:5) at async Job.doExecute (/home/runner/work/_actions/release-drafter/release-drafter/v5/dist/index.js:30793:18) { name: 'AggregateError', event: { id: '7135962636', name: 'pull_request', payload: { action: 'edited', changes: { body: { from: '## General Changes\r\n' + '\r\n' + '1. Adds serde and pickling `__reduce__` impl for `PyDaftConfig` so it can be sent over the wire in Ray\r\n' + '2. Parametrizes our runners (both PyRunner and RayRunner) with a `DaftConfig` object upon initialization. We thus also have to make it illegal from the user-API to set the configs once the runners have been instantiated (i.e. when any creation/execution of dataframes has occurred)\r\n' + '3. I also consolidated our `_RUNNER` and `_DAFT_CONTEXT` global singletons in the `daft.context` module -- the runner now resides inside the `DaftContext` object\r\n' + '\r\n' + '## RayRunner changes\r\n' + '\r\n' + '1. For every call to `@ray.remote`, ensure that we accept a daft_config as the first arg and call `set_config` as the first action\r\n' + '4. When the `RayRunner` is initialized, we call a `ray.put` to put the DaftConfig into the cluster store that objectref. This objectref is also passed around to various other datastructures that may potentially call `ray.remote` functions, such as the `RayPartitionSet` and `RayMaterializedResult` so that they can correctly parametrize their calls with the same config.' } }, number: 1707, organization: { avatar_url: 'https://avatars.githubusercontent.com/u/98941975?v=4', description: 'Eventual Computing', events_url: 'https://api.github.com/orgs/Eventual-Inc/events', hooks_url: 'https://api.github.com/orgs/Eventual-Inc/hooks', id: 98941975, issues_url: 'https://api.github.com/orgs/Eventual-Inc/issues', login: 'Eventual-Inc', members_url: 'https://api.github.com/orgs/Eventual-Inc/members{/member}', node_id: 'O_kgDOBeW8Fw', public_members_url: 'https://api.github.com/orgs/Eventual-Inc/public_members{/member}', repos_url: 'https://api.github.com/orgs/Eventual-Inc/repos', url: 'https://api.github.com/orgs/Eventual-Inc' }, pull_request: { _links: { comments: { href: 'https://api.github.com/repos/Eventual-Inc/Daft/issues/1707/comments' }, commits: { href: 'https://api.github.com/repos/Eventual-Inc/Daft/pulls/1707/commits' }, html: { href: 'https://github.com/Eventual-Inc/Daft/pull/1707' }, issue: { href: 'https://api.github.com/repos/Eventual-Inc/Daft/issues/1707' }, review_comment: { href: 'https://api.github.com/repos/Eventual-Inc/Daft/pulls/comments{/number}' }, review_comments: { href: 'https://api.github.com/repos/Eventual-Inc/Daft/pulls/1707/comments' }, self: { href: 'https://api.github.com/repos/Eventual-Inc/Daft/pulls/1707' }, statuses: { href: 'https://api.github.com/repos/Eventual-Inc/Daft/statuses/018748ebf2e2b407190f02a95aa28ae7de16a871' } }, active_lock_reason: null, additions: 204, assignee: null, assignees: [], author_association: 'CONTRIBUTOR', auto_merge: null, base: { label: 'Eventual-Inc:main', ref: 'main', repo: { allow_auto_merge: true, allow_forking: true, allow_merge_commit: false, allow_rebase_merge: false, allow_squash_merge: true, al