Tell us about the problem you're trying to solve

Our team is trying to use Zuora source to pull the data and persist it on s3 for latter processing, however the Data Query Live API used by default has a limitation which we are continuously hitting:

2021-10-11 11:29:05 ERROR Query failed (#20211011_112826_12620_ac93g): Input Rows for invoice exceeded limit (1000000), QUERY: select * from invoice where updateddate >= TIMESTAMP '2021-10-10 00:00:00.000000 +00:00' and updateddate <= TIMESTAMP '2021-11-09 00:00:00.000000 +00:00' order by updateddate asc 

They have an Unlimited option with this API however that has a 12 hour delay which is not ideal for us because our customers couldn't wait that long to have the data for the previous day.

To sum up the problem we would like to solve is having a way to export a bigger amount of data from Zuora without hitting the API limits and adding a substantial delay.

Describe the solution you’d like

It would be nice to expand the current Zuora source by adding the ability to use its AQuA API, which is capable to export any amount of data.

Describe the alternative you’ve considered or used

I found that the regular REST API has an export option which supports similar functionality, however it's not supporting all data sources, objects, and fields in Export ZOQL queries.