Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does bulk upsert data import support? #250

Open
melin opened this issue Dec 28, 2023 · 0 comments
Open

Does bulk upsert data import support? #250

melin opened this issue Dec 28, 2023 · 0 comments

Comments

@melin
Copy link

melin commented Dec 28, 2023

Bulk appended import data. If the primary key exists, update the data. How to import data efficiently?
The following operations can be performed in postgresql. Does sqlserver have similar features?

CREATE TABLE if not exists ${tempTableName} (LIKE ${tableName} INCLUDING defaults)

COPY $table FROM STDIN WITH (NULL '\\N', FORMAT CSV, DELIMITER E'${fieldDelimiter}'

insert into tableName (column1.....) select  * from  tempTableName on conflict (id) DO UPDATE SET (......)

https://github.com/melin/datatunnel/blob/master/connectors/jdbc/src/main/scala/com/superior/datatunnel/plugin/jdbc/support/CopyHelper.scala

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant