Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Standardized Marketing Models #146

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

fivetran-avinash
Copy link
Contributor

@fivetran-avinash fivetran-avinash commented Aug 30, 2024

PR Overview

This PR will address the following Issue/Feature: Standardized Marketing Models project

This PR will result in the following new package version: v0.19.0

Not a breaking change but this feature update is significant enough to warrant a versioning bump.

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Feature Updates

  • Addition of the following standardized marketing models.
model description
hubspot__marketing_campaigns_standardized This model constructs a comprehensive, denormalized analytical table that enables reporting on campaign metrics from your marketing platform. It’s designed to align with the schema of the *_marketing_campaigns_standardized model found in Hubspot, Marketo, Iterable, Klaviyo, Mailchimp, Qualtrics, Salesforce Marketing Cloud, and Twilio. To see the kinds of insights this model can generate, explore example visualizations in the Fivetran Marketing Model Streamlit App. Visit the app for more details.
hubspot__marketing_contacts_standardized This model constructs a comprehensive, denormalized analytical table that enables reporting on contacts from your marketing platform. It’s designed to align with the schema of the *_marketing_contacts_standardized model found in Hubspot, Marketo, Iterable, Klaviyo, Mailchimp, Qualtrics, Salesforce Marketing Cloud, and Twilio. To see the kinds of insights this model can generate, explore example visualizations in the Fivetran Marketing Model Streamlit App. Visit the app for more details.
hubspot__marketing_events_standardized This model constructs a comprehensive, denormalized analytical table that enables reporting on events from your marketing platform. It’s designed to align with the schema of the *_marketing_events_standardized model found in Hubspot, Marketo, Iterable, Klaviyo, Mailchimp, Qualtrics, Salesforce Marketing Cloud, and Twilio. To see the kinds of insights this model can generate, explore example visualizations in the Fivetran Marketing Model Streamlit App. Visit the app for more details.
  • These models construct comprehensive, denormalized analytical tables that enables reporting on campaigns, contacts, and events from your marketing platform. It’s designed to align with the schema of the *__line_item_enhanced model found in Hubspot, Marketo, Iterable, Klaviyo, Mailchimp, Qualtrics, Salesforce Marketing Cloud, and Twilio, offering standardized reporting across various marketing platforms. To see the kinds of insights this model can generate, explore example visualizations in the Fivetran Marketing Model Streamlit App. Visit the app for more details.
    • This model is currently disabled by default. You may enable it by setting the hubspot__standardized_marketing_model_enabled as true in your dbt_project.yml.

Under the Hood:

  • Added consistency tests within integration_tests for the hubspot__contacts, hubspot__email_campaigns and hubspot__email_sends models.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • [NA] dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

  • [NA] The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

Can confirm the new schema is being pulled through on a dbt run.

Screenshot 2024-08-30 at 4 19 50 AM

Validation tests were passing on both development.

Screenshot 2024-08-30 at 3 57 07 AM

As well as integration tests.
Screenshot 2024-08-29 at 3 59 13 PM

If you had to summarize this PR in an emoji, which would it be?

📧

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash thanks for working through this PR. I didn't review the documentation since we'll likely update that before release.

However, I did review the standardized models and have a few initial comments. Let me know if you want to sync on these at all.

Comment on lines 5 to 6
{% set emails_enabled = fivetran_utils.enabled_vars(['hubspot_marketing_enabled', 'hubspot_email_event_enabled', 'hubspot_email_event_sent_enabled']) %}
{% set engagements_enabled = fivetran_utils.enabled_vars(['hubspot_sales_enabled', 'hubspot_engagement_enabled','hubspot_engagement_contact_enabled']) %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these variables used at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are not. Removed.

total_deferrals,
total_deliveries,
total_opens,
cast(null as {{ dbt.type_int() }}) as total_unsubscribes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not available? It looks like we should have the information to generate unsubscribes from the reference table.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They were available, but took some rejiggering of the models to make this work. You can see the updated CHANGELOG for details (will add more in the Height ticket).

total_drops,
total_forwards,
total_opens,
cast(null as {{ dbt.type_int() }}) as total_unsubscribes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we should have this information in the underlying table. Is this not accessible?

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the above comment.

forwards as total_forwards,
opens as total_opens,
spam_reports as total_spam_reports,
cast(null as {{ dbt.type_int() }}) as total_unsubscribes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment about why these are not able to be used if it seems they are available in the underlying table.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the above comment.

was_delivered,
was_forwarded,
was_opened,
cast(null as {{ dbt.type_boolean()}}) was_unsubscribed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above comment. It looks like this should be available?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the above comment.

Copy link
Contributor Author

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-joemarkiewicz This PR is ready for re-review, see Height for more thorough notes on changes made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants