Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can I create schema with where condition like this? #285

Closed
phsophea101 opened this issue May 23, 2022 · 6 comments
Closed

can I create schema with where condition like this? #285

phsophea101 opened this issue May 23, 2022 · 6 comments

Comments

@phsophea101
Copy link

phsophea101 commented May 23, 2022

PGSync version: 2.1.9

Postgres version: 14

Elasticsearch version: 7.14.2

Redis version: 7.0.0

Python version: 3.7

Problem Description:
can I create schema with where condition like this?

SELECT * FROM users WHERE user_status = 'active';

[
    {
        "database": "db_name",
        "index": "user-index",
        "nodes": {
            "table": "product",
            "columns": [
                "id","name","desc","status"
            ],
            "children": [
                {
                    "table": "users",
                    "columns": [
                    ],
                    "relationship": {
                        "variant": "scalar",
                        "type": "one_to_one",
                        "foreign_key": {
                            "child": ["product_id"],
                            "parent": ["id"]
                        }
                    }
                }
            ]
        }
    }
]

I mean, how can I add WHERE user_status = 'active' in schema.

thank you in advnace

Error Message (if any):



@toluaina
Copy link
Owner

You would need to use the plugin feature for this.
Here is an example and documentation

example code:

class MyFilterPlugin(plugin.Plugin):
    """Example Filter plugin."""

    name: str = "Filter"

    def transform(self, doc: dict, **kwargs) -> dict:
        """Demonstrates how to perform a filter."""
        doc_id = kwargs["_id"]
        doc_index = kwargs["_index"]
        if doc["id"] == 1:
            return None

        return doc

@toluaina toluaina pinned this issue May 23, 2022
@phsophea101
Copy link
Author

@toluaina how can I add PYTHONPATH? sorry I am newbie in python.

@toluaina
Copy link
Owner

export PYTHONPATH=$PYTHONPATH:/this/is/the/path/to/plugin

This is further documented here
You can add a ltemporary log/print statement to the plugin to ensure it is getting called when you run pgsync

@phsophea101
Copy link
Author

I already made like this:

# echo $PYTHONPATH
/plugins
# ls
documents  plugins  runserver.sh  wait-for-it.sh
# cd plugins
# ls
__init__.py  activestatusplugin.py

but still got error:

pgsync-1         | Traceback (most recent call last):
pgsync-1         |   File "/usr/local/bin/bootstrap", line 70, in <module>
pgsync-1         |     main()
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
pgsync-1         |     return self.main(*args, **kwargs)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
pgsync-1         |     rv = self.invoke(ctx)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
pgsync-1         |     return ctx.invoke(self.callback, **ctx.params)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
pgsync-1         |     return __callback(*args, **kwargs)
pgsync-1         |   File "/usr/local/bin/bootstrap", line 60, in main
pgsync-1         |     document, verbose=verbose, repl_slots=False, **kwargs
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 94, in __init__
pgsync-1         |     self.validate(repl_slots=repl_slots)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 175, in validate
pgsync-1         |     root: Node = self.tree.build(self.nodes)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/pgsync/node.py", line 251, in build
pgsync-1         |     raise NodeAttributeError(f"Unknown node attribute(s): {attrs}")
pgsync-1         | pgsync.exc.NodeAttributeError: "Unknown node attribute(s): {'plugins'}"
pgsync-1         |  0:00:00.038423 (0.04 sec)
pgsync-1         | Traceback (most recent call last):
pgsync-1         |   File "/usr/local/bin/pgsync", line 7, in <module>
pgsync-1         |     sync.main()
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
pgsync-1         |     return self.main(*args, **kwargs)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
pgsync-1         |     rv = self.invoke(ctx)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
pgsync-1         |     return ctx.invoke(self.callback, **ctx.params)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
pgsync-1         |     return __callback(*args, **kwargs)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 1248, in main
pgsync-1         |     sync: Sync = Sync(document, verbose=verbose, **kwargs)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 94, in __init__
pgsync-1         |     self.validate(repl_slots=repl_slots)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 175, in validate
pgsync-1         |     root: Node = self.tree.build(self.nodes)
pgsync-1         |   File "/usr/local/lib/python3.7/site-packages/pgsync/node.py", line 251, in build
pgsync-1         |     raise NodeAttributeError(f"Unknown node attribute(s): {attrs}")
pgsync-1         | pgsync.exc.NodeAttributeError: "Unknown node attribute(s): {'plugins'}"

please help me @toluaina

@toluaina
Copy link
Owner

@phsophea101

I think you would be better served dropping me a message on discord taina#4505.
It appears you have an error in your schema.json with the way you have declared the plugins

@phsophea101
Copy link
Author

@toluaina Its working now,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants