⭐ Improve parameterized query support - fixes #793 #794

mvanderlee · 2024-02-22T00:13:26Z

Fixes #793
The implemented tests speak for themselves, but here is an extract:

subquery = (
    Query.from_(self.table_efg)
    .select(self.table_efg.fiz, self.table_efg.buz)
    .where(self.table_efg.buz == 'buz')
)

q = (
    Query.from_(self.table_abc)
    .join(subquery)
    .on(self.table_abc.bar == subquery.buz)
    .select(self.table_abc.foo, subquery.fiz)
    .where(self.table_abc.bar == 'bar')
)

parameter = NamedParameter()
sql = q.get_sql(parameter=parameter)
self.assertEqual(
    'SELECT "abc"."foo","sq0"."fiz" FROM "abc" JOIN (SELECT "fiz","buz" FROM "efg" WHERE "buz"=:param1)'
    ' "sq0" ON "abc"."bar"="sq0"."buz" WHERE "abc"."bar"=:param2',
    sql,
)
self.assertEqual({'param1': 'buz', 'param2': 'bar'}, parameter.get_parameters())

This would allow the user to easily plug into SQLAlchemy for example:

parameter = NamedParameter()
session.execute(text(q.get_sql(parameter=parameter)), **parameter.get_parameters())

wd60622 · 2024-02-22T06:06:28Z

pypika/terms.py

+    @property
+    def placeholder(self):
+        if callable(self._placeholder):
+            return self._placeholder(None)


how does this work on None?

callable Returns False

But I presume you mean how will the self_placeholder(None) work with None.
Good point, it wouldn't atm with the default generators.

I will fix it now.

wd60622 · 2024-02-22T06:12:28Z

Thanks for the PR! Will take a deeper look later

coveralls · 2024-02-28T15:11:15Z

coverage: 98.392% (-0.006%) from 98.398%
when pulling 9ba1241 on mvanderlee:mvanderlee/parametized_query
into 53a77eb on kayak:master.

mvanderlee · 2024-03-12T08:13:21Z

Thanks for the PR! Will take a deeper look later

@wd60622 Do you think you'll have time this week?

wd60622

Looks really good.
Just some comments on type hints, raising errors, and combining logic

pypika/terms.py

wd60622 · 2024-03-13T19:14:25Z

pypika/terms.py

+def named_placeholder_gen(idx: int) -> str:
+    return f'param{idx + 1}'
+
+
 class Parameter(Term):
    is_aggregate = None

    def __init__(self, placeholder: Union[str, int]) -> None:


Type hint here doesn't include callable? Is that correct still?

Correct, only ListParameter and DictParameter (and classes that inherit from them) support callable.
The base parameter does not.

wd60622 · 2024-03-13T19:17:01Z

pypika/terms.py

 class Parameter(Term):
    is_aggregate = None

    def __init__(self, placeholder: Union[str, int]) -> None:
        super().__init__()
-        self.placeholder = placeholder
+        self._placeholder = placeholder


Might be good to use a setter here. With that, implementing a check for the correct type can happen on initialization if put in the setter

I'm not sure I follow how to do this pythonically? Could you share some pseudo code to show what you mean?

@mvanderlee I believe @wd60622 was referring to the following:

@property def placeholder(self): return self._placeholder @placeholder.setter def placeholder(self, placeholder): # can add checks here if needed self._placeholder = placeholder

Right, which adds no value and isn't pythonic. This isn't Java/C#.

It would also break parameters. After a parameter has been created and used, any changes to the placeholder will break queries. Which is why I've made it private with a getter only.

I haven't changed the type, so any type checks would have to be part of a separate PR.

I would disagree with you on it not being Pythonic, as it’s a native language feature.

Also, I was trying to be helpful, but given your receptiveness to discussion and tone, I won’t proceed.

Good luck.

Sorry for my tone @danielenricocahall it'd been a rough week and I lashed out.

As for pythonic. This video does a good job at explaining language feature vs pythonic.
Raymond Hettinger - Beyond PEP 8 -- Best practices for beautiful intelligible code - https://www.youtube.com/watch?v=wf-BqAjZb8M

wd60622 · 2024-03-13T19:18:36Z

Do you have any thoughts as well? @AzisK

mvanderlee · 2024-04-05T15:02:33Z

@wd60622 @AzisK any update on this?
It should go hand in hand with #792

AzisK · 2024-04-26T15:21:31Z

Could we also have this functionality described in pypika docs?

AzisK · 2024-04-26T15:27:26Z

The tests look really nice. What about adding PyformatParameter in the tests as well?

reasv · 2024-09-07T21:16:10Z

I just wanted to say that I'm going to fork your branch because this feature is really useful, and I'm using it in my own project to help me build a custom high level query language: https://github.com/reasv/panoptikon/blob/master/src/panoptikon/db/pql/build_query.py

I'm resorting to making my own fork of PyPika and merging a bunch of these PRs because it doesn't seem PyPika is getting much merged lately and there are open bugs that otherwise block development: d7a4054

larsschwegmann · 2024-09-27T12:04:12Z

Any progress on getting this merged? :)

AzisK · 2024-09-27T12:13:11Z

Hello, I asked @mvanderlee for a test for PyformatParameter and I believe it would be good to have it. Otherwise I could merge this PR

larsschwegmann · 2024-09-27T12:38:58Z

I forked @mvanderlee 's fork and added a test for PyformatParameter: Commit

AzisK · 2024-09-27T19:21:47Z

@larsschwegmann could you make a PR? Is it better for me to push this change to this commit? Probably

larsschwegmann · 2024-09-28T10:10:06Z

If it's possible, I guess it would be best if you could just add the commit to this PR, just for the sake of documentation. Then you'll still have all the discussion in the PR that was merged 😄
If not, I can create a PR. Let me know what works best for you.

mvanderlee · 2024-09-28T10:23:04Z

Thank you @larsschwegmann I've added it to the branch!

AzisK · 2024-10-10T16:38:16Z

I could merge but linting fails. It's not a big deal but it would be good to lint before. Also unit tests for other Python versions fail but it's probably a bug of Github actions which is super annoying. It seems that they fail due to a race condition

mvanderlee · 2024-10-11T07:58:29Z

@AzisK I'll run the linter, but it's an issue in the master branch. The linter is failing on lines that were not altered by this PR. I see the same errors with other PRs.

mvanderlee · 2024-10-11T08:16:57Z

The affected code wasn't in this fork yet, so I rebased in order to fix it.

AzisK · 2024-10-21T15:43:33Z

All seems good, I can merge it. Thanks for the work

mvanderlee requested a review from a team as a code owner February 22, 2024 00:13

mvanderlee mentioned this pull request Feb 22, 2024

🐛 Fixes #791 #792

Open

wd60622 reviewed Feb 22, 2024

View reviewed changes

wd60622 reviewed Mar 13, 2024

View reviewed changes

larsschwegmann mentioned this pull request Sep 30, 2024

add upstream changes from kayak/pypika #794 - Improve parameterized q… tortoise/pypika-tortoise#11

Merged

mvanderlee and others added 5 commits October 11, 2024 10:14

Add parameterized query support

d565832

Revert base Parameter constructor back to it's original signature

b82d8e2

Fix a few typehints and make code more DRY

a5c981c

add test for PyformatParameter

e324436

fix linting issues

9ba1241

mvanderlee force-pushed the mvanderlee/parametized_query branch from 2ae30df to 9ba1241 Compare October 11, 2024 08:15

AzisK merged commit 4072bfb into kayak:master Oct 21, 2024
5 checks passed

AzisK assigned mvanderlee Nov 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⭐ Improve parameterized query support - fixes #793 #794

⭐ Improve parameterized query support - fixes #793 #794

mvanderlee commented Feb 22, 2024 •

edited

Loading

wd60622 Feb 22, 2024

mvanderlee Feb 22, 2024

wd60622 commented Feb 22, 2024

coveralls commented Feb 28, 2024 •

edited

Loading

mvanderlee commented Mar 12, 2024

wd60622 left a comment

wd60622 Mar 13, 2024

mvanderlee Mar 13, 2024

wd60622 Mar 13, 2024

mvanderlee Mar 13, 2024

danielenricocahall Apr 19, 2024

mvanderlee Apr 19, 2024

danielenricocahall Apr 19, 2024

mvanderlee Apr 24, 2024

wd60622 commented Mar 13, 2024

mvanderlee commented Apr 5, 2024

AzisK commented Apr 26, 2024 •

edited

Loading

AzisK commented Apr 26, 2024

reasv commented Sep 7, 2024

larsschwegmann commented Sep 27, 2024

AzisK commented Sep 27, 2024

larsschwegmann commented Sep 27, 2024 •

edited

Loading

AzisK commented Sep 27, 2024

larsschwegmann commented Sep 28, 2024

mvanderlee commented Sep 28, 2024

AzisK commented Oct 10, 2024

mvanderlee commented Oct 11, 2024

mvanderlee commented Oct 11, 2024

AzisK commented Oct 21, 2024

⭐ Improve parameterized query support - fixes #793 #794

⭐ Improve parameterized query support - fixes #793 #794

Conversation

mvanderlee commented Feb 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wd60622 commented Feb 22, 2024

coveralls commented Feb 28, 2024 • edited Loading

mvanderlee commented Mar 12, 2024

wd60622 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wd60622 commented Mar 13, 2024

mvanderlee commented Apr 5, 2024

AzisK commented Apr 26, 2024 • edited Loading

AzisK commented Apr 26, 2024

reasv commented Sep 7, 2024

larsschwegmann commented Sep 27, 2024

AzisK commented Sep 27, 2024

larsschwegmann commented Sep 27, 2024 • edited Loading

AzisK commented Sep 27, 2024

larsschwegmann commented Sep 28, 2024

mvanderlee commented Sep 28, 2024

AzisK commented Oct 10, 2024

mvanderlee commented Oct 11, 2024

mvanderlee commented Oct 11, 2024

AzisK commented Oct 21, 2024

mvanderlee commented Feb 22, 2024 •

edited

Loading

coveralls commented Feb 28, 2024 •

edited

Loading

AzisK commented Apr 26, 2024 •

edited

Loading

larsschwegmann commented Sep 27, 2024 •

edited

Loading