Skip to content

Conversation

@bazarnov
Copy link
Contributor

@bazarnov bazarnov commented Aug 11, 2021

What

#5293 - Source Hubspot fails in normalization step

How

  • added additional field_value verification for an empty string to api.py, if so - set the field_value to be zero, instead of empty string.
  • edited test_field_type_converting.py unit-test to simulate the issue of empty string inside numeric (float) type

NOTE:
This fix potentially could resolve the issue with Bad Double Value when using this connector along with Destination BigQuery for data replication. As far as BigQuery does not accept simple null values, and could not accept empty strings as numeric values, it requires additional data preprocessing before inject the data into the table.

Pre-merge Checklist

Updating a connector

Community member or Airbyter

  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions
  • Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

Oleksandr Bazarnov added 2 commits August 11, 2021 11:28
@github-actions github-actions bot added the area/connectors Connector related issues label Aug 11, 2021
@bazarnov bazarnov changed the title 🐛 Source HubSpot: Fix for bad double values when sync with BigQuery destination 🐛 Source HubSpot: Fix empty string inside number / float datatype Aug 11, 2021
@bazarnov bazarnov requested review from antixar and midavadim August 11, 2021 09:24
@github-actions github-actions bot added the area/documentation Improvements or additions to documentation label Aug 11, 2021
@bazarnov
Copy link
Contributor Author

bazarnov commented Aug 11, 2021

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1119528170
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1119528170

@bazarnov bazarnov self-assigned this Aug 11, 2021
@bazarnov bazarnov linked an issue Aug 11, 2021 that may be closed by this pull request
@jrhizor jrhizor temporarily deployed to more-secrets August 11, 2021 09:34 Inactive
@bazarnov bazarnov requested a review from midavadim August 11, 2021 17:24
@jrhizor jrhizor temporarily deployed to more-secrets August 12, 2021 14:15 Inactive
@bazarnov
Copy link
Contributor Author

bazarnov commented Aug 12, 2021

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1124238148
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1124238148

@jrhizor jrhizor temporarily deployed to more-secrets August 12, 2021 14:24 Inactive
@bazarnov
Copy link
Contributor Author

bazarnov commented Aug 12, 2021

/publish connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1124341394
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1124341394

@jrhizor jrhizor temporarily deployed to more-secrets August 12, 2021 14:55 Inactive
@bazarnov bazarnov merged commit 6f56833 into master Aug 12, 2021
@bazarnov bazarnov deleted the bazarnov/5293-hubspot-bad-double-values branch August 12, 2021 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/connectors Connector related issues area/documentation Improvements or additions to documentation

4 participants