[source-hubspot] Add support for Hubspot Deal Splits

[jtruty]

What

This adds support for a new Hubspot API - Deal Splits. Hubspot recently enabled a similar API for this endpoint similar to Deals (where we can search by last modified date) so this PR follows the Hubspot Deal format.
Documentation for this is here: https://developers.hubspot.com/docs/api/crm/search#:~:text=/crm/v3/objects/deal_split,%2C%C2%A0hs_lastmodifieddate%2C%C2%A0hs_object_id, this was added as a new object that is available via search, so we can access very similarly to how we access the deals object. Incremental syncs will work the same way - by tracking the hs_lastmodifieddate. The actual data returned will look similar to a standard Hubspot deal, with a few differences:

• The addition of a hs_deal_split_percentage field that is set to 1 for non-split deals
• For split deals, there will be a separate deal_split record for each configured split. Each record associated with a single deal will have a unique hs_object_id, but the same hs_deal_id. In addition, there will be new fields hs_deal_split_percentage, hs_deal_split_amount that represent the split percentage and amounts for this split, respectively. The hubspot_owner_id field on the split will be associated with the Hubspot owner for the split record (and not necessarily the same deal owner).

Using the REST search api with filters, a request looks like this:

POST https://api.hubapi.com/crm/v3/objects/deal_split/search/ { "filterGroups": [{ "filters": [{ "propertyName": "hs_lastmodifieddate", "operator": "GT", "value": "2023-07-01T20:23:28.968Z" }, { "propertyName": "hs_deal_split_percentage", "operator": "NEQ", "value": 1 }] }], "sorts": [{ "propertyName": "hs_deal_id" }], "properties": ["hs_deal_split_percentage","hs_deal_split_amount", "hs_deal_id", "hubspot_owner_id"] } 

And a sample response is:

{ "total": 1, "results": [ { "id": "1234569", "properties": { "hs_createdate": "2023-01-26T02:42:31.339Z", "hs_deal_id": "11414699925", "hs_deal_split_amount": "640.0", "hs_deal_split_percentage": "0.2", "hs_lastmodifieddate": "2023-01-27T18:58:47.514Z", "hs_object_id": "12334595", "hubspot_owner_id": "1883747" }, "createdAt": "2023-01-26T02:42:31.339Z", "updatedAt": "2023-01-27T18:58:47.514Z", "archived": false } ] } 

This matches the format of the existing Hubspot CRM search streams in Airbyte, with some additional fields.

How

Add a new DealSplit stream as a Searchable stream, similar to the Hubspot Deal stream.

Review guide

airbyte-integrations/connectors/source-hubspot/source_hubspot/streams.py

User Impact

User can select an extra stream to sync - deal_split stream

Can this PR be safely reverted and rolled back?

• YES 💚
• NO ❌

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Aug 20, 2024 9:29pm

[jtruty]

@jtruty Thanks for the excellent documentation of your work. It looks good to me, but can you add some unit tests? You can probably add the new stream around here, but it would also be nice if we had a safety net for this new stream inside the integrations folder.

Also, there is a conflict for md file.

Thanks!

@aldogonzalez8 I fixed the conflicts, bumped the version, and added the deal split stream to the main list of streams to check. I took a look at the integrations folder, but didn't see a similar test for a hubspot search stream to base one on. Maybe we can add a ticket to add an integration test for the deals stream? That should help with covering many related ones.

[aldogonzalez8]

https://developers.hubspot.com/docs/api/crm/search#:~:text=/crm/v3/objects/deal_split,%2C%C2%A0hs_lastmodifieddate%2C%C2%A0hs_object_id

That's fair. Would you happen to have the instructions for creating a deal split in the Hubspot UI? I want to see if I can run regression and get a record or two for the new stream.

[jtruty]

That's fair. Would you happen to have the instructions for creating a deal split in the Hubspot UI? I want to see if I can run regression and get a record or two for the new stream.

@aldogonzalez8: this should cover it: https://knowledge.hubspot.com/records/split-deal-credit-among-users, let me know if you still need a hand!

[aldogonzalez8]

That's fair. Would you happen to have the instructions for creating a deal split in the Hubspot UI? I want to see if I can run regression and get a record or two for the new stream.

@aldogonzalez8: this should cover it: https://knowledge.hubspot.com/records/split-deal-credit-among-users, let me know if you still need a hand!

Okay, this is interesting. The acceptance test will fail because we don't have records in the testing account for deal splits, so we have to either ignore or add records. Depending on the effort, I will try to do this in the afternoon and get back to you on this.

[aldogonzalez8]

@jtruty can you update the stream name in the catalogs? I think it should be plural, I was unable to find the stream name till I changed it to deal_splits

[jtruty]

@jtruty can you update the stream name in the catalogs? I think it should be plural, I was unable to find the stream name till I changed it to deal_splits

Sure thing, done.

[aldogonzalez8]

@jtruty Can you help me to add this record as a new line in expected_records? Be sure to left a new line (enter) after add this.

{"stream":"deal_splits","data":{"id":"357038921479","properties":{"hs_all_accessible_team_ids":null,"hs_all_assigned_business_unit_ids":null,"hs_all_owner_ids":"65568071","hs_all_team_ids":null,"hs_created_by_user_id":12282590.0,"hs_createdate":"2024-08-20T17:14:10.498000+00:00","hs_deal_amount":1000.0,"hs_deal_amount_in_home_currency":1000.0,"hs_deal_close_date":"2024-08-01T20:45:24.748000+00:00","hs_deal_create_date":"2024-08-20T17:11:30.127000+00:00","hs_deal_currency_code":null,"hs_deal_forecast_category":null,"hs_deal_forecast_probability":null,"hs_deal_id":21586456222.0,"hs_deal_owner_id":52550153.0,"hs_deal_pipeline_id":"default","hs_deal_split_amount":500.0,"hs_deal_split_amount_in_home_currency":500.0,"hs_deal_split_forecast_amount":500.0,"hs_deal_split_percentage":0.5,"hs_deal_split_projected_amount":100.0,"hs_deal_split_projected_amount_in_home_currency":100.0,"hs_deal_split_type":null,"hs_deal_stage":"appointmentscheduled","hs_deal_stage_probability":0.2,"hs_lastmodifieddate":"2024-08-20T20:45:30.385000+00:00","hs_merged_object_ids":null,"hs_object_id":357038921479.0,"hs_object_source":"CRM_UI","hs_object_source_detail_1":null,"hs_object_source_detail_2":null,"hs_object_source_detail_3":null,"hs_object_source_id":"userId:12282590","hs_object_source_label":"CRM_UI","hs_object_source_user_id":12282590.0,"hs_read_only":null,"hs_shared_team_ids":null,"hs_shared_user_ids":null,"hs_unique_creation_key":null,"hs_updated_by_user_id":12282590.0,"hs_user_ids_of_all_notification_followers":null,"hs_user_ids_of_all_notification_unfollowers":null,"hs_user_ids_of_all_owners":"23660227","hs_was_imported":null,"hubspot_owner_assigneddate":"2024-08-20T17:14:10.498000+00:00","hubspot_owner_id":"65568071","hubspot_team_id":null},"createdAt":"2024-08-20T17:14:10.498Z","updatedAt":"2024-08-20T20:45:30.385Z","archived":false,"properties_hs_all_accessible_team_ids":null,"properties_hs_all_assigned_business_unit_ids":null,"properties_hs_all_owner_ids":"65568071","properties_hs_all_team_ids":null,"properties_hs_created_by_user_id":12282590.0,"properties_hs_createdate":"2024-08-20T17:14:10.498000+00:00","properties_hs_deal_amount":1000.0,"properties_hs_deal_amount_in_home_currency":1000.0,"properties_hs_deal_close_date":"2024-08-01T20:45:24.748000+00:00","properties_hs_deal_create_date":"2024-08-20T17:11:30.127000+00:00","properties_hs_deal_currency_code":null,"properties_hs_deal_forecast_category":null,"properties_hs_deal_forecast_probability":null,"properties_hs_deal_id":21586456222.0,"properties_hs_deal_owner_id":52550153.0,"properties_hs_deal_pipeline_id":"default","properties_hs_deal_split_amount":500.0,"properties_hs_deal_split_amount_in_home_currency":500.0,"properties_hs_deal_split_forecast_amount":500.0,"properties_hs_deal_split_percentage":0.5,"properties_hs_deal_split_projected_amount":100.0,"properties_hs_deal_split_projected_amount_in_home_currency":100.0,"properties_hs_deal_split_type":null,"properties_hs_deal_stage":"appointmentscheduled","properties_hs_deal_stage_probability":0.2,"properties_hs_lastmodifieddate":"2024-08-20T20:45:30.385000+00:00","properties_hs_merged_object_ids":null,"properties_hs_object_id":357038921479.0,"properties_hs_object_source":"CRM_UI","properties_hs_object_source_detail_1":null,"properties_hs_object_source_detail_2":null,"properties_hs_object_source_detail_3":null,"properties_hs_object_source_id":"userId:12282590","properties_hs_object_source_label":"CRM_UI","properties_hs_object_source_user_id":12282590.0,"properties_hs_read_only":null,"properties_hs_shared_team_ids":null,"properties_hs_shared_user_ids":null,"properties_hs_unique_creation_key":null,"properties_hs_updated_by_user_id":12282590.0,"properties_hs_user_ids_of_all_notification_followers":null,"properties_hs_user_ids_of_all_notification_unfollowers":null,"properties_hs_user_ids_of_all_owners":"23660227","properties_hs_was_imported":null,"properties_hubspot_owner_assigneddate":"2024-08-20T17:14:10.498000+00:00","properties_hubspot_owner_id":"65568071","properties_hubspot_team_id":null},"emitted_at":1724187597772} 

Also, for abnormal_state.json please help me to add this new entry in the array.

 { "type": "STREAM", "stream": { "stream_descriptor": { "name": "deal_splits" }, "stream_state": { "updatedAt": "2200-01-01T00:00:00.000000Z" } } }, 

I think is all pending to have Acceptance passing.

[jtruty]

@jtruty Can you help me to add this record as a new line in expected_records? Be sure to left a new line (enter) after add this.

{"stream":"deal_splits","data":{"id":"357038921479","properties":{"hs_all_accessible_team_ids":null,"hs_all_assigned_business_unit_ids":null,"hs_all_owner_ids":"65568071","hs_all_team_ids":null,"hs_created_by_user_id":12282590.0,"hs_createdate":"2024-08-20T17:14:10.498000+00:00","hs_deal_amount":1000.0,"hs_deal_amount_in_home_currency":1000.0,"hs_deal_close_date":"2024-08-01T20:45:24.748000+00:00","hs_deal_create_date":"2024-08-20T17:11:30.127000+00:00","hs_deal_currency_code":null,"hs_deal_forecast_category":null,"hs_deal_forecast_probability":null,"hs_deal_id":21586456222.0,"hs_deal_owner_id":52550153.0,"hs_deal_pipeline_id":"default","hs_deal_split_amount":500.0,"hs_deal_split_amount_in_home_currency":500.0,"hs_deal_split_forecast_amount":500.0,"hs_deal_split_percentage":0.5,"hs_deal_split_projected_amount":100.0,"hs_deal_split_projected_amount_in_home_currency":100.0,"hs_deal_split_type":null,"hs_deal_stage":"appointmentscheduled","hs_deal_stage_probability":0.2,"hs_lastmodifieddate":"2024-08-20T20:45:30.385000+00:00","hs_merged_object_ids":null,"hs_object_id":357038921479.0,"hs_object_source":"CRM_UI","hs_object_source_detail_1":null,"hs_object_source_detail_2":null,"hs_object_source_detail_3":null,"hs_object_source_id":"userId:12282590","hs_object_source_label":"CRM_UI","hs_object_source_user_id":12282590.0,"hs_read_only":null,"hs_shared_team_ids":null,"hs_shared_user_ids":null,"hs_unique_creation_key":null,"hs_updated_by_user_id":12282590.0,"hs_user_ids_of_all_notification_followers":null,"hs_user_ids_of_all_notification_unfollowers":null,"hs_user_ids_of_all_owners":"23660227","hs_was_imported":null,"hubspot_owner_assigneddate":"2024-08-20T17:14:10.498000+00:00","hubspot_owner_id":"65568071","hubspot_team_id":null},"createdAt":"2024-08-20T17:14:10.498Z","updatedAt":"2024-08-20T20:45:30.385Z","archived":false,"properties_hs_all_accessible_team_ids":null,"properties_hs_all_assigned_business_unit_ids":null,"properties_hs_all_owner_ids":"65568071","properties_hs_all_team_ids":null,"properties_hs_created_by_user_id":12282590.0,"properties_hs_createdate":"2024-08-20T17:14:10.498000+00:00","properties_hs_deal_amount":1000.0,"properties_hs_deal_amount_in_home_currency":1000.0,"properties_hs_deal_close_date":"2024-08-01T20:45:24.748000+00:00","properties_hs_deal_create_date":"2024-08-20T17:11:30.127000+00:00","properties_hs_deal_currency_code":null,"properties_hs_deal_forecast_category":null,"properties_hs_deal_forecast_probability":null,"properties_hs_deal_id":21586456222.0,"properties_hs_deal_owner_id":52550153.0,"properties_hs_deal_pipeline_id":"default","properties_hs_deal_split_amount":500.0,"properties_hs_deal_split_amount_in_home_currency":500.0,"properties_hs_deal_split_forecast_amount":500.0,"properties_hs_deal_split_percentage":0.5,"properties_hs_deal_split_projected_amount":100.0,"properties_hs_deal_split_projected_amount_in_home_currency":100.0,"properties_hs_deal_split_type":null,"properties_hs_deal_stage":"appointmentscheduled","properties_hs_deal_stage_probability":0.2,"properties_hs_lastmodifieddate":"2024-08-20T20:45:30.385000+00:00","properties_hs_merged_object_ids":null,"properties_hs_object_id":357038921479.0,"properties_hs_object_source":"CRM_UI","properties_hs_object_source_detail_1":null,"properties_hs_object_source_detail_2":null,"properties_hs_object_source_detail_3":null,"properties_hs_object_source_id":"userId:12282590","properties_hs_object_source_label":"CRM_UI","properties_hs_object_source_user_id":12282590.0,"properties_hs_read_only":null,"properties_hs_shared_team_ids":null,"properties_hs_shared_user_ids":null,"properties_hs_unique_creation_key":null,"properties_hs_updated_by_user_id":12282590.0,"properties_hs_user_ids_of_all_notification_followers":null,"properties_hs_user_ids_of_all_notification_unfollowers":null,"properties_hs_user_ids_of_all_owners":"23660227","properties_hs_was_imported":null,"properties_hubspot_owner_assigneddate":"2024-08-20T17:14:10.498000+00:00","properties_hubspot_owner_id":"65568071","properties_hubspot_team_id":null},"emitted_at":1724187597772} 

Also, for abnormal_state.json please help me to add this new entry in the array.

 { "type": "STREAM", "stream": { "stream_descriptor": { "name": "deal_splits" }, "stream_state": { "updatedAt": "2200-01-01T00:00:00.000000Z" } } }, 

I think is all pending to have Acceptance passing.

Both added.

[aldogonzalez8]

@jtruty Can you help me to add this record as a new line in expected_records? Be sure to left a new line (enter) after add this.

{"stream":"deal_splits","data":{"id":"357038921479","properties":{"hs_all_accessible_team_ids":null,"hs_all_assigned_business_unit_ids":null,"hs_all_owner_ids":"65568071","hs_all_team_ids":null,"hs_created_by_user_id":12282590.0,"hs_createdate":"2024-08-20T17:14:10.498000+00:00","hs_deal_amount":1000.0,"hs_deal_amount_in_home_currency":1000.0,"hs_deal_close_date":"2024-08-01T20:45:24.748000+00:00","hs_deal_create_date":"2024-08-20T17:11:30.127000+00:00","hs_deal_currency_code":null,"hs_deal_forecast_category":null,"hs_deal_forecast_probability":null,"hs_deal_id":21586456222.0,"hs_deal_owner_id":52550153.0,"hs_deal_pipeline_id":"default","hs_deal_split_amount":500.0,"hs_deal_split_amount_in_home_currency":500.0,"hs_deal_split_forecast_amount":500.0,"hs_deal_split_percentage":0.5,"hs_deal_split_projected_amount":100.0,"hs_deal_split_projected_amount_in_home_currency":100.0,"hs_deal_split_type":null,"hs_deal_stage":"appointmentscheduled","hs_deal_stage_probability":0.2,"hs_lastmodifieddate":"2024-08-20T20:45:30.385000+00:00","hs_merged_object_ids":null,"hs_object_id":357038921479.0,"hs_object_source":"CRM_UI","hs_object_source_detail_1":null,"hs_object_source_detail_2":null,"hs_object_source_detail_3":null,"hs_object_source_id":"userId:12282590","hs_object_source_label":"CRM_UI","hs_object_source_user_id":12282590.0,"hs_read_only":null,"hs_shared_team_ids":null,"hs_shared_user_ids":null,"hs_unique_creation_key":null,"hs_updated_by_user_id":12282590.0,"hs_user_ids_of_all_notification_followers":null,"hs_user_ids_of_all_notification_unfollowers":null,"hs_user_ids_of_all_owners":"23660227","hs_was_imported":null,"hubspot_owner_assigneddate":"2024-08-20T17:14:10.498000+00:00","hubspot_owner_id":"65568071","hubspot_team_id":null},"createdAt":"2024-08-20T17:14:10.498Z","updatedAt":"2024-08-20T20:45:30.385Z","archived":false,"properties_hs_all_accessible_team_ids":null,"properties_hs_all_assigned_business_unit_ids":null,"properties_hs_all_owner_ids":"65568071","properties_hs_all_team_ids":null,"properties_hs_created_by_user_id":12282590.0,"properties_hs_createdate":"2024-08-20T17:14:10.498000+00:00","properties_hs_deal_amount":1000.0,"properties_hs_deal_amount_in_home_currency":1000.0,"properties_hs_deal_close_date":"2024-08-01T20:45:24.748000+00:00","properties_hs_deal_create_date":"2024-08-20T17:11:30.127000+00:00","properties_hs_deal_currency_code":null,"properties_hs_deal_forecast_category":null,"properties_hs_deal_forecast_probability":null,"properties_hs_deal_id":21586456222.0,"properties_hs_deal_owner_id":52550153.0,"properties_hs_deal_pipeline_id":"default","properties_hs_deal_split_amount":500.0,"properties_hs_deal_split_amount_in_home_currency":500.0,"properties_hs_deal_split_forecast_amount":500.0,"properties_hs_deal_split_percentage":0.5,"properties_hs_deal_split_projected_amount":100.0,"properties_hs_deal_split_projected_amount_in_home_currency":100.0,"properties_hs_deal_split_type":null,"properties_hs_deal_stage":"appointmentscheduled","properties_hs_deal_stage_probability":0.2,"properties_hs_lastmodifieddate":"2024-08-20T20:45:30.385000+00:00","properties_hs_merged_object_ids":null,"properties_hs_object_id":357038921479.0,"properties_hs_object_source":"CRM_UI","properties_hs_object_source_detail_1":null,"properties_hs_object_source_detail_2":null,"properties_hs_object_source_detail_3":null,"properties_hs_object_source_id":"userId:12282590","properties_hs_object_source_label":"CRM_UI","properties_hs_object_source_user_id":12282590.0,"properties_hs_read_only":null,"properties_hs_shared_team_ids":null,"properties_hs_shared_user_ids":null,"properties_hs_unique_creation_key":null,"properties_hs_updated_by_user_id":12282590.0,"properties_hs_user_ids_of_all_notification_followers":null,"properties_hs_user_ids_of_all_notification_unfollowers":null,"properties_hs_user_ids_of_all_owners":"23660227","properties_hs_was_imported":null,"properties_hubspot_owner_assigneddate":"2024-08-20T17:14:10.498000+00:00","properties_hubspot_owner_id":"65568071","properties_hubspot_team_id":null},"emitted_at":1724187597772} 

Also, for abnormal_state.json please help me to add this new entry in the array.

 { "type": "STREAM", "stream": { "stream_descriptor": { "name": "deal_splits" }, "stream_state": { "updatedAt": "2200-01-01T00:00:00.000000Z" } } }, 

I think is all pending to have Acceptance passing.

Both added.

Perfect, I will leave acceptance tests running. Regression looks good, as the major difference is the new stream found by the discover command

[aldogonzalez8]

APPROVED