🎉Source Twilio: Added state checkpoint interval (#16110)

* Added state checkpoint interval * Bumped version * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

Author: lazebnyi

airbyte-config/init/src/main/resources/seed/source_definitions.yaml

@@ -1036,11 +1036,11 @@
 - name: Twilio
  sourceDefinitionId: b9dc6155-672e-42ea-b10d-9f1f1fb95ab1
  dockerRepository: airbyte/source-twilio
- dockerImageTag: 0.1.7
+ dockerImageTag: 0.1.8
  documentationUrl: https://docs.airbyte.io/integrations/sources/twilio
  icon: twilio.svg
  sourceType: api
- releaseStage: alpha
+ releaseStage: beta
 - name: Typeform
  sourceDefinitionId: e7eff203-90bf-43e5-a240-19ea3056c474
  dockerRepository: airbyte/source-typeform

airbyte-config/init/src/main/resources/seed/source_specs.yaml

@@ -10280,7 +10280,7 @@
  oauthFlowOutputParameters:
  - - "token"
  - - "key"
-- dockerImage: "airbyte/source-twilio:0.1.7"
+- dockerImage: "airbyte/source-twilio:0.1.8"
  spec:
  documentationUrl: "https://docs.airbyte.io/integrations/sources/twilio"
  connectionSpecification:

airbyte-integrations/connectors/source-twilio/Dockerfile

@@ -12,5 +12,5 @@ COPY main.py ./
 ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
-LABEL io.airbyte.version=0.1.7
+LABEL io.airbyte.version=0.1.8
 LABEL io.airbyte.name=airbyte/source-twilio

airbyte-integrations/connectors/source-twilio/README.md

@@ -79,7 +79,7 @@ docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integrat
 Make sure to familiarize yourself with [pytest test discovery](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery) to know how your test files and methods should be named.
 First install test dependencies into your virtual environment:
 ```
-pip install .[tests]
+pip install .'[tests]'
 ```
 ### Unit Tests
 To run unit tests locally, from the connector directory run:

airbyte-integrations/connectors/source-twilio/source_twilio/streams.py

@@ -87,17 +87,18 @@ def custom_transform_function(original_value: Any, field_schema: Mapping[str, An
  try:
  return pendulum.from_format(original_value, "ddd, D MMM YYYY HH:mm:ss ZZ").in_timezone("UTC").to_iso8601_string()
  except ValueError:
- # Twilio API returns datetime in two formats:
- #  - RFC2822, like "Fri, 11 Dec 2020 04:28:40 +0000";
- #  - ISO8601, like "2020-12-11T04:29:09Z".
- # If `ValueError` exception was raised this means that datetime was already in ISO8601 format and there
- # is no need in transforming anything.
+ """Twilio API returns datetime in two formats:
+ - RFC2822, like "Fri, 11 Dec 2020 04:28:40 +0000";
+ - ISO8601, like "2020-12-11T04:29:09Z".
+ If `ValueError` exception was raised this means that datetime was already in ISO8601 format and there
+ is no need in transforming anything."""
  pass
  return original_value
 
 
 class IncrementalTwilioStream(TwilioStream, IncrementalMixin):
  time_filter_template = "YYYY-MM-DD HH:mm:ss[Z]"
+ state_checkpoint_interval = 1000
 
  def __init__(self, start_date: str = None, lookback_window: int = 0, **kwargs):
  super().__init__(**kwargs)

docs/integrations/sources/twilio.md

@@ -80,7 +80,8 @@ For more information, see [the Twilio docs for rate limitations](https://support
 
 | Version | Date | Pull Request | Subject |
 |:--------|:------------|:---------------------------------------------------------|:--------------------------------------------------------------------------------------------------------|
-| 0.1.7 | 2022-08-26 | [15972](https://github.com/airbytehq/airbyte/pull/15972) | Shift start date for stream if it exceeds 400 days |
+| 0.1.8 | 2022-08-29 | [16110](https://github.com/airbytehq/airbyte/pull/16110) | Add state checkpoint interval |
+| 0.1.7 | 2022-08-26 | [15972](https://github.com/airbytehq/airbyte/pull/15972) | Shift start date for stream if it exceeds 400 days |
 | 0.1.6 | 2022-06-22 | [14000](https://github.com/airbytehq/airbyte/pull/14000) | Update Records stream schema and align tests with connectors' best practices |
 | 0.1.5 | 2022-06-22 | [13896](https://github.com/airbytehq/airbyte/pull/13896) | Add lookback window parameters to fetch messages with a rolling window and catch status updates |
 | 0.1.4 | 2022-04-22 | [12157](https://github.com/airbytehq/airbyte/pull/12157) | Use Retry-After header for backoff |