[FEAT] Improved projection folding. #1820
Annotations
2 errors
Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}
{
name: 'HttpError',
id: '6252687511',
status: 422,
response: {
url: 'https://api.github.com/repos/Eventual-Inc/Daft/releases/120823407',
status: 422,
headers: {
'access-control-allow-origin': '*',
'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
connection: 'close',
'content-length': '195',
'content-security-policy': "default-src 'none'",
'content-type': 'application/json; charset=utf-8',
date: 'Wed, 20 Sep 2023 18:33:34 GMT',
'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
server: 'GitHub.com',
'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
vary: 'Accept-Encoding, Accept, X-Requested-With',
'x-accepted-github-permissions': 'contents=write',
'x-content-type-options': 'nosniff',
'x-frame-options': 'deny',
'x-github-api-version-selected': '2022-11-28',
'x-github-media-type': 'github.v3; format=json',
'x-github-request-id': '889F:01A6:80FCE3:1057CD2:650B3AFD',
'x-ratelimit-limit': '1000',
'x-ratelimit-remaining': '980',
'x-ratelimit-reset': '1695238296',
'x-ratelimit-resource': 'core',
'x-ratelimit-used': '20',
'x-xss-protection': '0'
},
data: {
message: 'Validation Failed',
errors: [
{
resource: 'Release',
code: 'invalid',
field: 'target_commitish'
}
],
documentation_url: 'https://docs.github.com/rest/releases/releases#update-a-release'
}
},
request: {
method: 'PATCH',
url: 'https://api.github.com/repos/Eventual-Inc/Daft/releases/120823407',
headers: {
accept: 'application/vnd.github.v3+json',
'user-agent': 'probot/12.2.5 octokit-core.js/3.5.1 Node.js/16.20.1 (linux; x64)',
authorization: 'token [REDACTED]',
'content-type': 'application/json; charset=utf-8'
},
body: '{"body":"## Changes\\n\\n## ✨ New Features\\n\\n- [FEAT] Implement .dt.year/month/day for timestamp types @jaychia (#1385)\\n- [FEAT] [New Query Planner] Add support for fsspec filesystems to new query planner. @clarkzinzow (#1357)\\n- [FEAT] Common subexpression elimination in Projection construction @xcharleslin (#1347)\\n\\n## 👾 Bug Fixes\\n\\n- [BUG] Fix partition spec bugs from old query planner @xcharleslin (#1372)\\n\\n## 📖 Documentation\\n\\n- [FEAT] Implement .dt.year/month/day for timestamp types @jaychia (#1385)\\n\\n## 🧰 Maintenance\\n\\n- [CHORE] Add stubs and improve comments for pyo3-exposed abstractions, + driveby type/bug fixes. @clarkzinzow (#1377)\\n- [CHORE] add retries for broken link checker @samster25 (#1378)\\n- [CHORE] pin azure-storage-blob due to breaking new version @samster25 (#1373)\\n- [CHORE] [New Query Planner] Misc. user-facing error tweaks to improve UX. @clarkzinzow (#1358)\\n","draft":true,"prerelease":false,"name":"v0.1.18","tag_name":"v0.1.18","target_commitish":"refs/pull/1374/merge"}',
request: {}
},
event: {
id: '6252687511',
name: 'pull_request',
payload: {
action: 'edited',
changes: {
body: {
from: "While looking at the projection folding behaviour in the old query planner, I noticed there were some projections that could be folded that it didn't fold. \r\n" +
'\r\n' +
'For example, (a + 1) -> (a as c) could be folded into (a + 1 as c), but the old query planner does not do that:\r\n' +
'\r\n' +
'```\r\n' +
'>>> daft.from_pydict({"a": [1, 2], "b": [3, 4]}).select((col("a")+1)).select(col("a").alias("c")).explain(show_optimized=True)\r\n' +
'2023-09-13 19:34:09.494 | INFO | daft.context:runner:87
|
HttpError: Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}
at /home/runner/work/_actions/release-drafter/release-drafter/v5/dist/index.js:8462:21
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async Job.doExecute (/home/runner/work/_actions/release-drafter/release-drafter/v5/dist/index.js:30793:18)
{
name: 'AggregateError',
event: {
id: '6252687511',
name: 'pull_request',
payload: {
action: 'edited',
changes: {
body: {
from: "While looking at the projection folding behaviour in the old query planner, I noticed there were some projections that could be folded that it didn't fold. \r\n" +
'\r\n' +
'For example, (a + 1) -> (a as c) could be folded into (a + 1 as c), but the old query planner does not do that:\r\n' +
'\r\n' +
'```\r\n' +
'>>> daft.from_pydict({"a": [1, 2], "b": [3, 4]}).select((col("a")+1)).select(col("a").alias("c")).explain(show_optimized=True)\r\n' +
'2023-09-13 19:34:09.494 | INFO | daft.context:runner:87 - Using PyRunner\r\n' +
'┌─Projection\r\n' +
'│ output=[col(a) AS c]\r\n' +
"│ partitioning={'by': None, 'num_partitions': 1, 'scheme': PartitionScheme.Unknown}\r\n" +
'│\r\n' +
'├──Projection\r\n' +
'│ output=[col(a) + lit(1)]\r\n' +
"│ partitioning={'by': None, 'num_partitions': 1, 'scheme': PartitionScheme.Unknown}\r\n" +
'│\r\n' +
'└──InMemoryScan\r\n' +
' output=[col(a), col(b)]\r\n' +
" cache_id='fc55df21728b4bf9bb1d6143559af9d7'\r\n" +
" partitioning={'by': None, 'num_partitions': 1, 'scheme': PartitionScheme.Unknown}\r\n" +
'```\r\n' +
'\r\n' +
'The rules of the old projection folding were that:\r\n' +
'\r\n' +
'- A projection P with a child projection C can be folded together if P only references columns that are no computation in C (no-ops/aliases).\r\n' +
'\r\n' +
'However, there are projections with computation in C that can be folded into P. Specifically, they can be safely folded if all columns in C are referenced at most once in P. Then, the expressions in C can be directly substituted for the column references in P without changing any execution semantics.\r\n' +
'\r\n' +
'This PR implements improved projection folding with this new rule. The new query planner will fold the example above into:\r\n' +
'\r\n' +
'```\r\n' +
'* Project: col(a) + lit(1) AS c\r\n' +
'|\r\n' +
'* Output schema = a (Int64)\r\n' +
'```\r\n' +
'\r\n' +
'TODO: tests\r\n' +
'\r\n' +
'\r\n' +
'\r\n'
}
},
number: 1374,
organization: {
avatar_url: 'https://avatars.githubusercontent.com/u/98941975?v=4',
description: 'Eventual Computing',
events_url: 'https://api.github.com/orgs/Eventual-Inc/events',
hooks_url: 'https://api.github.com/orgs/Eventual-Inc/hooks',
id: 98941975,
issues_url: 'https://api.github.com/orgs/Eventual-Inc/issues',
login: 'Eventual-Inc',
members_url: 'https://api.github.com/orgs/Eventual-Inc/members{/member}',
node_id: 'O_kgDOBeW8Fw',
public_members_url: 'https://api.github.com/orgs/Eventual-Inc/public_members{/member}',
repos_url: 'https://api.github.com/orgs/Eventual-Inc/repos',
url: 'https://api.github.com/orgs/Eventual-Inc'
},
pull_request: {
_links: {
comments: {
href: 'https://api.github.com/repos/Eventual-Inc/Daft/issues/1374/comments'
},
commits: {
href: 'https://api.github.com/repos/Eventual-Inc/Daft/pulls/1374/commits'
},
html: { href: 'https://github.com/Eventual-Inc/Daft
|
The logs for this run have expired and are no longer available.
Loading