Releases: fivetran/dbt_hubspot_source
v0.21.0 dbt_hubspot_source
This release includes the following updates:
Schema Changes
3 total changes • 3 possible breaking changes
Data Models | Change Type | Old Behavior | New Behavior | Notes |
---|---|---|---|---|
stg_hubspot__deal_pipeline_stage | Column Renamed | is_closed_won |
is_closed |
Column renamed to more accurately match the source data. The renamed is_closed column is generated via a coalesce between the previous closed_won source column and the newly added is_closed column. This renamed column represents whether a deal is closed, regardless of its label as “Closed Won” or “Closed Lost.” You can still use the pipeline_stage_label column to differentiate between "Closed Won" and "Closed Lost" stages. (PR #146) |
stg_hubspot__ticket_pipeline |
Column datatype | ticket_pipeline_id (int ) |
ticket_pipeline_id (string ) |
Reverted ticket pipeline identifier back to the original string format to prevent datatype errors in compilation. (PR #151) |
stg_hubspot__ticket_pipeline_stage |
Column datatype | ticket_pipeline_id (int )ticket_pipeline_stage_id (int ) |
ticket_pipeline_id (string )ticket_pipeline_stage_id (string ) |
Reverted ticket pipeline identifiers back to their original string format to prevent datatype errors in compilation. (PR #151) |
Under the Hood
- Updated seed files to validate that the new identifiers compile as expected in both the source and transform packages. (PR #151)
- Documented the column updates and deprecations in the respective yml files. (PR #146)
- Updated the
deal_pipeline_stage_data
seed to include theis_closed
column. (PR #146) - Introduced the generate-docs github workflow for consistent docs generation. (PR #146)
- Updated the maintainer pull request template. (PR #147)
- Updated conditions in
.github/workflows/auto-release.yml
. (PR #147) - Added
.github/workflows/generate-docs.yml
. (PR #146) - Added
+docs: show: False
tointegration_tests/dbt_project.yml
. (PR #146) - Migrated
flags
(e.g.,send_anonymous_usage_stats
,use_colors
) fromsample.profiles.yml
tointegration_tests/dbt_project.yml
. (PR #146) - Updated
maintainer_pull_request_template.md
with improved checklist. (PR #146) - Refreshed README tag block (PR #147):
- Left-aligned and positioned below the H1 title.
- Updated
.gitignore
to exclude additional DBT, Python, and system artifacts. (PR #147)
Full Changelog: v0.20.0...v0.21.0
v0.20.0 dbt_hubspot_source
PR #144 includes the following updates:
Breaking Change for dbt Core < 1.9.6
Note: This is not relevant to Fivetran Quickstart users.
Migrated freshness
from a top-level source property to a source config
in alignment with recent updates from dbt Core. This will resolve the following deprecation warning that users running dbt >= 1.9.6 may have received:
[WARNING]: Deprecated functionality
Found `freshness` as a top-level property of `hubspot` in file
`models/src_hubspot.yml`. The `freshness` top-level property should be moved
into the `config` of `hubspot`.
IMPORTANT: Users running dbt Core < 1.9.6 will not be able to utilize freshness tests in this release or any subsequent releases, as older versions of dbt will not recognize freshness as a source config
and therefore not run the tests.
If you are using dbt Core < 1.9.6 and want to continue running HubSpot freshness tests, please elect one of the following options:
- (Recommended) Upgrade to dbt Core >= 1.9.6
- Do not upgrade your installed version of the
hubspot_source
package. Pin your dependency on v0.19.0 in yourpackages.yml
file. - Utilize a dbt override to overwrite the package's
hubspot
source and apply freshness via the previous release top-level property route. This will require you to copy and paste the entirety of the previous releasesrc_hubspot.yml
file and add anoverrides: hubspot_source
property.
Under the Hood
- Updates to ensure integration tests use latest version of dbt.
Full Changelog: v0.19.0...v0.20.0
v0.19.0 dbt_hubspot_source
PR #142 includes the following updates:
Breaking Changes
- Introduced new sources and their associated staging models to support expanded HubSpot data coverage:
Source Table | Staging Model | Enablement Variable(s) - Default is true unless otherwise mentioned |
---|---|---|
contact_form_submission |
stg_hubspot__contact_form_submission |
hubspot_contact_form_enabled hubspot_marketing_enabled |
engagement_communication |
stg_hubspot__engagement_communication |
hubspot_engagement_communication_enabled (default: false) hubspot_engagement_enabled hubspot_sales_enabled |
form |
stg_hubspot__form |
hubspot_contact_form_enabled hubspot_marketing_enabled |
owner_team |
stg_hubspot__owner_team |
hubspot_team_enabled |
role |
stg_hubspot__role |
hubspot_role_enabled |
team |
stg_hubspot__team |
hubspot_team_enabled |
team_user |
stg_hubspot__team_user |
hubspot_team_user_enabled hubspot_team_enabled |
user |
stg_hubspot__user |
hubspot_role_enabled |
- See Step 4 of the README for more details on enabling/disabling sources.
- Updated the
owner
source andstg_hubspot__owner
staging model:- Added the
active_user_id
column. - Removed the requirement for
hubspot_sales_enabled
to be true since this logic now applies to multiple objects (e.g. tickets and deals) downstream.
- Added the
Documentation
- Added column-level descriptions for all new models and fields.
Under the Hood
- Added seeds for each new source.
- Added
get_*_columns
macros for each new source.
Full Changelog: v0.18.1...v0.19.0
v0.18.1 dbt_hubspot_source
PR #140 includes the following updates:
Fixes
- Removes the uniqueness test for
stg_hubspot__property
on_fivetran_id
and adds aunique_combination_of_columns
test using_fivetran_id
andhubspot_object
. This was necessary to account for a primary key change to thePROPERTY
table within HubSpot connector.- See the
PROPERTY
table within the HubSpot Marketing Hub Schema ERD for more details about the updated composite key.
- See the
Contributors
Full Changelog: v0.18.0...v0.18.1
v0.18.0 dbt_hubspot_source
Deprecations
- Select fields have been deprecated from the
stg_hubspot__contact_list
model. The full removal date is planned for May 10th, 2025. Follow Deprecation Issue #138 for more details and updates around the planned sunsetting of the fields. (#137)- The deprecated fields include the following:
is_deletable
is_dynamic
metadata_error
metadata_last_processing_state_change_at
metadata_last_size_change_at
metadata_processing
metadata_size
portal_id
- The deprecated fields include the following:
Breaking Changes
- Select fields have been added to the
stg_hubspot__contact_list
model. (#137)- The newly added fields include the following:
created_by_id
object_type_id
processing_status
processing_type
list_version
filters_updated_at
- The newly added fields include the following:
Documentation
- All deprecated fields have been documented as notice of deprecation in the respective yml documents. (#137)
- All added fields have been documented. (#137)
- Corrected references to connectors and connections in the README. (#136)
Under the Hood
- Added a deprecation tracker GitHub issue template to ensure deprecated fields are appropriately tracked for full removal in a future planned update. (#137)
Full Changelog: v0.17.0...v0.18.0
v0.17.0 dbt_hubspot_source
Breaking Change (--full-refresh
required after upgrading)
-
Introduced a new
category
column to the below models. This association field differentiates records by either HUBSPOT_DEFINED (default label) or USER_DEFINED (custom label) and was introduced to the Hubspot connector in October 2024. See the connector release notes for more.stg_hubspot__deal_company
stg_hubspot__deal_contact
stg_hubspot__engagement_company
stg_hubspot__engagement_contact
stg_hubspot__engagement_deal
stg_hubspot__ticket_company
stg_hubspot__ticket_contact
stg_hubspot__ticket_deal
stg_hubspot__ticket_engagement
-
Added explicit casting of
pipeline_id
to string instg_hubspot__deal_pipeline_stage
to ensure successful downstream joins.
Bug Fixes
- Updated the unique key to include the new
category
field for the following models in order for the uniqueness tests to pass:stg_hubspot__deal_contact
stg_hubspot__deal_company
Under the Hood
- Updated the respective seed files,
get_[source_table]_column
macros, and documentation of the mentioned models to include thecategory
field. - Removed
property_hs_timestamp
andproperty_hs_createdate
field casting for theengagement_meeting
source table in theintegration_tests/dbt_project.yml
file since they are not present in the seed data.
Full Changelog: v0.16.0...v0.17.0
v0.16.0 dbt_hubspot_source
PR #129 includes the following updates:
Breaking Changes
- Switched from using the
fivetran_utils.remove_prefix_from_columns
macro to thehubspot_source.remove_duplicate_and_prefix_from_columns
macro for whenhubspot__pass_through_all_columns
is enabled and you are passing through all columns in thestg_hubspot__company
,stg_hubspot__contact
, andstg_hubspot__deal
models. This also ensures the source fields passed through are all quoted from the onset. This is a breaking change because this macro can remove duplicate fields, resulting in an impact to your schema.
Bug Fixes
- Introduced hubspot-specific version of the
fivetran_utils.pass_through_columns
macro titledhubspot_add_pass_through_columns
, which introduces quoting around the source fields being brought in as passthrough columns. This will ensure that your warehouse reads the sql correctly, particularly if the field contains special characters or syntax. This is now used in the respectiveget_<>_columns
macros of the following models:stg_hubspot__company
stg_hubspot__contact
stg_hubspot__deal
stg_hubspot__ticket
Under the Hood
- Updated seed data to include fields with special syntax in order to test the above changes.
Full Changelog: v0.15.0...v0.16.0
v0.15.0 dbt_hubspot_source
PR #126 includes the following updates:
🚨 Breaking Changes 🚨
- Added field
_fivetran_end
to macroget_ticket_property_history_columns()
to ensure the column is available for use downstream.- While this should not affect most users, this will add a new column
_fivetran_end
tostg_hubspot__ticket_property_history
if you do not have_fivetran_end
in your sourceTICKET_PROPERTY_HISTORY
table.
- While this should not affect most users, this will add a new column
Documentation
- Update documentation to include
_fivetran_end
under modelstg_hubspot__ticket_property_history
.
Under the hood
- Updated the maintainer PR template to the current format.
- Included auto-releaser GitHub Actions workflow to automate future releases.
- Added integration testing for Databricks SQL Warehouse.
Full Changelog: v0.14.1...v0.15.0
v0.14.1 dbt_hubspot_source
PR #125 includes the following updates:
Features
- Adds the
stg_hubspot__merged_deal
model. Downstream, this model is used to remove stale deals and aggregate them to the deal they have been merged into. This is disabled by default as not every user has themerged_deal
table.- To enable this model and filter out merged deals downstream, make sure the following is set to
true
in your dbt_project.yml:hubspot_sales_enabled
,hubspot_deal_enabled
, andhubspot_merged_deal_enabled
. See Step 4 of the README for more details.
- To enable this model and filter out merged deals downstream, make sure the following is set to
Full Changelog: v0.14.0...v0.14.1
v0.14.0 dbt_hubspot_source
PR #122 includes the following updates:
Features
-
Added the following staging models, along with documentation and tests:
stg_hubspot__property
stg_hubspot__property_option
- These tables can be disabled by setting
hubspot_property_enabled: False
in your dbt_project.yml vars. See Step 4 of the README for more details.
-
When including a passthrough
property_hs_*
column, you now have the option to include the corresponding, human-readable label in the staging models.- The above-mentioned
property
tables are required for this feature. If you do not have them and have to disable them, unfortunately you will not be able to use this feature. - See the Adding property label section of the README for instructions on how to enable this feature!
- We recommend being selective with the label columns you add. As you add more label columns, your run time will increase due to the underlying logic requirements.
- This update applies to models:
stg_hubspot__company
stg_hubspot__contact
stg_hubspot__deal
stg_hubspot__ticket
- The above-mentioned
Bug fixes
- Updated macro
remove_duplicate_and_prefix_from_columns
to accommodate incoming custom column names containing characters such as-
or$
that are not permitted. The resulting column name will have these characters removed or replaced in itsstg_*
model. - Removed extra comma from
stg_hubspot__ticket
, which was causing compilation issues when passing through all columns.
Full Changelog: v0.13.0...v0.14.0