Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow the user to specify the column name for snowplow_incremental logic #70

Closed
emielver opened this issue Apr 4, 2022 · 2 comments
Closed
Assignees
Labels
category:macros Related to the macros in the package. type:enhancement New features or improvements to existing features.

Comments

@emielver
Copy link
Contributor

emielver commented Apr 4, 2022

Describe the feature

With the snowplow_incremental logic, users aren't able to specify which column should be used to see which data has been processed. The is_run_with_new_events macro is currently hardcoded to look for start_tstamp, however technically this could be any timestamp column. We should allow users to specify which column they want to point to to see which events have been processed already.

Proposed Solution

In macros/utils/is_run_with_new_events.sql we should replace start_tstamp with a parameter such as {{ start_timestamp }} which should take on a default value of start_tstamp but could be specified by the user in the is_run_with_new_events macro call. We should also create a test to ensure that this is working as expected with custom column names.

@emielver emielver self-assigned this Apr 4, 2022
@rlh1994 rlh1994 added type:enhancement New features or improvements to existing features. category:macros Related to the macros in the package. labels Dec 13, 2022
@rlh1994
Copy link
Contributor

rlh1994 commented Dec 13, 2022

@emielver We'll cover this in Jan, but is the idea they can choose what is set into start_tstamp and recorded into the manifest?

@rlh1994 rlh1994 closed this as completed Sep 19, 2023
@emielver
Copy link
Contributor Author

Closed in #144

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:macros Related to the macros in the package. type:enhancement New features or improvements to existing features.
Projects
None yet
Development

No branches or pull requests

2 participants